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value  is  now  the  goal  of  all  negotiated  Government  procurements.  One  measure  of  contracting 

officer  effectiveness  is  to  examine  protest  decisions  handed  down  by  the  General  Accounting 

Office  (GAO).  This  thesis  examines  all  protest  decisions  of  best  value  awards  from  1997.  The 

research  reveals  a  sustainment  (success)  rate  of  19.44  percent  for  1997.  GAO's  published 

sustainment  rate  for  all  protests  is  12  percent  for  the  same  year. 

Best  value  sustainments  (21  total)  are  first  categorized  in  this  thesis  by  agency 
improprieties  in  the  evaluation  of  a  tradeoff  element  (e.g.  past  performance,  technical  merit, 
cost/price,  or  labor  qualifications)  or  improper  pre-  or  post-award  changes.  The  sustainments  are 
then  thoroughly  analyzed  to  reveal  pitfalls  which  contracting  officers  must  avoid  to  preclude 
protest  sustainment. 

These  pitfalls  are  then  incorporated  into  a  final  analysis  where  they  are  merged  with  the 
contracting  process  (acquisition  planning,  solicitation,  source  evaluation/selection,  negotiation, 
award,  contract  administration)  and  examined  for  greater  clarity.  Examples  of  the  common 
pitfalls  which  resulted  in  sustainment  in  1997  are  uncertainty  of  requirements,  poorly  crafted 
solicitations,  failure  to  follow  solicitations,  failure  to  use  all  relevant  facts,  failure  to  evaluate  total 
cost/price,  improper  cost/price  realism  analyses,  pre-award  solicitation  changes  without 
modification,  failure  to  hold  meaningful  discussions  when  required,  failure  to  support  contract 
award  with  narrative,  out-of-scope  post  award  changes,  and  contract  administrative  improprieties. 
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I.  INTRODUCTION 

A.         BACKGROUND 

Best  Value  is  the  driving  theme  in  the  newly  approved  rewrite  of  Federal 
Acquisition  Regulation  (FAR)  Part  15.  The  stated  purpose  of  this  was  to  infuse 
innovative  techniques  into  the  source  selection  process,  simplify  the  process,  and 
facilitate  contract  award  under  best  value  criteria.  [Ref.  1 :  p.  51224]  The  task  at  hand  is 
for  Federal  agency  contracting  officers  to  implement  best  value  source  selection  into 
Government  procurement  in  a  manner  consistent  with  Congress'  intentions.  One  method 
used  by  Congress  to  exert  its  right  to  influence  and  investigate  Government  acquisition  is 
through  the  system  of  award  protest.  Congress'  investigative  arm,  the  General 
Accounting  Office  (GAO)  provides  an  arena  for  contract  award  protests  and  renders  non- 
binding  but  influential  decisions  regarding  source  selection.  This  mechanism  is  the  result 
of  a  statutory  mandate  under  the  Competition  in  Contracting  Act  (CICA)  of  1984  tasking 
GAO  to  assess  the  reasonableness  of  source  selection  decisions  and  determine  whether  a 
procurement  has  been  conducted  in  accordance  with  all  applicable  statutes  and 
regulations  [Ref.  2].  The  overarching  goal  of  CICA  is  full  and  open  competition,  and  this 
Act  has  dominated  Government  contracting  for  over  a  decade.  GAO  protest  decisions  are 
a  key  indicator  of  Congress'  intent  in  the  domain  of  Government  procurement.  Federal 
agencies  of  the  Executive  Branch  should  look  to  such  decisions  for  messages  from  the 
Legislative  Branch. 


During  a  protest,  GAO  requires  an  agency  to  provide  certain  information 
consistent  with  FAR  documentation  requirements  so  that  it  may  be  established  whether 
the  evaluation  of  proposals  and  the  source  selection  decision  were  reasonable.  The  FAR 
has  always  required  agencies  to  document  the  relative  strengths,  deficiencies,  significant 
weakness,  and  risks  supporting  the  proposal  evaluation  and  source  selection.  However, 
the  FAR  Part  15  rewrite  recognizes  the  subjective  nature  inherent  in  best  value 
contracting.  Contracting  officers  must  now  balance  tradeoffs  in  an  effort  to  obtain  the 
best  value  for  the  Government.  Tradeoffs  are  appropriate  when  it  may  be  in  the  best 
interest  of  the  Government  to  consider  award  to  an  offeror  other  than  the  lowest  priced 
offeror  or  the  highest  technically  rated  offeror.  As  a  result,  the  FAR  Part  1 5  rewrite 
requires  that  in  addition  to  common  quantitative  measures  and  matrices  used  in  proposal 
evaluation,  the  agency  shall  document  rationale  for  any  business  judgments  and  tradeoffs 
including  supporting  narrative  [Ref.  1:  15.305,  15.308].     GAO  has  recently  upheld 
protests  because  adjectival  ratings  needed  to  be  accompanied  by  a  reasoned  explanation 
for  cost/non-cost  tradeoffs.  In  addition,  agency  contracting  officers  now  face  increased 
GAO  scrutiny  of  their  judgment  of  what  constitutes  best  value  in  addition  to  the  typical 
review  of  adjectival  ratings,  matrices,  technical  appraisals,  and  cost  evaluation. 
B.         OBJECTIVE 

The  goal  of  this  research  is  to  ascertain  whether  or  not  best  value  contracting  is 
more  susceptible  to  sustainable  protests.  The  thesis  focuses  on  contracting  officer 


judgment  in  source  selection  and  evaluates  the  difficulties  regarding  contract  award 

through  best  value  determination.  This  document  outlines  the  genesis  and  background  of 

best  value,  describes  laws  and  regulations  that  apply,  and  researches  the  146  Comptroller 

General  protest  decisions  from  1997.  Lessons  from  the  sustained  protests  of  1997  have 

been  developed  to  assist  contracting  officers  to  effectively  implement  best  value  in  source 

selection. 

C.         THE  RESEARCH  QUESTION 

The  primary  research  question  is:  "To  what  extent  does  the  exercise  of  Federal 
agency  contracting  officer  judgment,  in  best  value  source  selection,  result  in  a  higher  rate 
of  GAO  protest  sustainment  than  other  methods  of  source  selection?"  The  subsidiary 
research  questions  are: 

•  What  is  best  value,  the  genesis  of  the  concept,  the  evolution  of  this  type  of 
contracting,  current  implementation,  and  the  regulations  that  apply? 

•  Can  the  types  of  award  protests  and  subsequent  sustainments  be  categorized? 
If  so,  what  are  the  categories? 

•  Are  there  measures  that  a  contracting  officer  can  take  to  preclude  protests 
regarding  best  value  awards? 

•  How  may  contracting  officers  be  better  trained  regarding  best  value  awards? 


D.         SCOPE,  METHODOLOGY,  LIMITATIONS,  AND  ASSUMPTIONS 

This  study  focuses  on  the  current  nature  and  pitfalls  inherent  to  best  value 
contracting.  The  primary  concentration  of  study  consists  of  the  146  decisions  rendered 
by  the  Comptroller  General  in  response  to  protests  of  contract  award  under  best  value 
criteria.  Other  areas  of  study  include  a  literature  search  of  Government  regulations, 
magazine  articles,  commentary,  and  other  library  information  resources.    The 
methodology  consists  of  the  following: 

1 .  A  brief  examination  of  all  best  value  protests  to  identify  sustainments  and 
basic  statistics. 

2.  A  thorough  examination  of  the  sustained  protests. 

3.  Categorization  of  the  sustained  protests. 

4.  Analysis  of  the  sustained  protests. 

5.  Generation  of  recommendations  and  conclusions  regarding  the  sustained 
protests. 

This  research  is  limited  to  best  value  protests  that  allege  Federal  agency 
improprieties  in  the  determination  of  a  best  value  award  and/or  Federal  agency  actions 
that  undermine  an  otherwise  proper  best  value  contract  award.  The  primary  assumption 
in  this  study  is  that  the  reader  is  familiar  with  the  basic  rudiments,  types,  and  processes  of 
Government  contracting. 


E.         ORGANIZATION  OF  THE  THESIS 

The  following  chapters  in  this  thesis  document  the  significance  of  best  value  and 
examine  the  protests  involving  best  value  contracting  in  1997.  Chapter  II  is  a  review  of 
best  value  contracting  including  the  genesis,  evolution,  and  current  state  of  this  type  of 
acquisition.  Chapter  III  is  an  overall  study  of  all  best  value  protests  in  1997.  The  study 
includes  a  review  of  the  protest  system,  overall  number  of  protests,  and  the  overall 
success  of  protests  in  general  throughout  1997.  This  chapter  also  categorizes  the 
recurring  reasons  for  protest.  Chapters  IV,  V,  VI,  and  VII  review  and  analyze  the  best 
value  protests  that  were  sustained  due  to  Federal  agency  improprieties  in  the  evaluation 
of  past  performance,  technical  merit,  cost/price,  and  labor  qualifications  respectively. 
Chapter  VIII  reviews  and  analyzes  protests  that  were  sustained  due  to  improper  Federal 
agency  pre-  and  post-award  changes.  Chapter  IX  provides  a  final  analysis  and 
conclusion. 


II.         REVIEW  OF  BEST  VALUE  CONTRACTING 

A.  INTRODUCTION 

This  chapter  provides  background  into  the  concept  of  best  value.  The  purpose  of 
this  chapter  is  to  provide  the  reader  with  a  firm  grasp  of  the  concept  from  its  inception  to 
its  present  state.    Thorough  understanding  of  the  best  value  concept  provides  the  reader 
with  a  firm  foundation  of  knowledge  to  make  better  use  of  the  material  discussed  in  the 
remaining  chapters  of  the  thesis.  This  chapter  addresses  best  value  in  the  following 
manner.  First,  important  definitions  of  best  value  terms  are  outlined.  Second,  the  genesis 
of  the  best  value  concept  is  discussed.  Third,  the  evolution  of  best  value  is  detailed. 
Finally,  a  review  of  the  current  state  of  best  value  is  conducted. 

B.  DEFINITIONS 

The  following  definitions  are  derived  from  the  Federal  Acquisition  Regulation 
(FAR)  and  are  important  to  understand  fully  the  nature  and  current  state  of  best  value 
contracting  in  the  Federal  Government. 

1.  Best  Value 

The  expected  outcome  of  an  acquisition  that,  in  the  Government's  estimation, 
provides  the  greatest  overall  benefit  in  response  to  the  requirement.  [Ref.  1:  2.101] 

2.  Best  Value  Continuum 

An  agency  can  obtain  best  value  in  negotiated  acquisitions  by  using  any  one  or  a 
combination  of  source  selection  approaches.  In  different  types  of  acquisitions,  the 


2.  Best  Value  Continuum 

An  agency  can  obtain  best  value  in  negotiated  acquisitions  by  using  any  one  or  a 
combination  of  source  selection  approaches.  In  different  types  of  acquisitions,  the 
relative  importance  of  cost  or  price  may  vary.  The  choosing  of  the  source  selection 
approach  will  normally  relate  to  the  relative  importance  between  cost  and  non-cost 
factors.  [Ref.  1:  15.101] 

3.  Tradeoff  Process 

In  negotiated  procurement,  the  approach  most  commonly  used  to  achieve  a  best 
value  selection  is  a  tradeoff  process  where  dollars  are  traded  for  incremental  value  and 
cost  is  only  one  factor,  not  the  controlling  factor.  A  tradeoff  process  is  appropriate  when 
it  may  be  in  the  best  interest  of  the  Government  to  consider  award  to  other  than  the 
lowest  priced  offeror  or  other  than  the  highest  technically  rated  offeror.  [Ref.  1 :  15.101-1] 

4.  Lowest  Price-Technically  Acceptable 

The  lowest  price-technically  acceptable  process  is  appropriate  when  best  value  is 
expected  to  result  from  selection  of  the  technically  acceptable  proposal  with  the  lowest 
evaluated  price.  [Ref.  1:15.101-2] 
C.         GENESIS  OF  BEST  VALUE 

There  is  no  definitive  starting  point  to  the  concept  of  best  value.  It  is  clear  that 
businesses  have  historically  sought  to  purchase  goods  and  services  advantageous  to  their 
pursuit  of  profit.  The  Government  is  not  motivated  by  profit;  however,  increased 


emphasis  has  been  placed  on  the  efficiency  of  the  acquisition  process  over  the  past  1 5 
years.  The  genesis  of  best  value  contracting  in  Government  acquisition  can  be  explored 
through  study  of  the  Government's  efforts  to  reform  the  acquisition  process  and  adopt 
commercial  practices.  [Ref.  7] 

1.  Concept  Origin 

During  the  mid-1980's  a  series  of  acquisition  reform  measures  began  that  affect 
every  aspect  of  Government  contracting  to  this  day.  Most  of  these  measures  addressed 
best  value  either  directly  or  indirectly.  The  first  move  toward  the  current  state  of  best 
value  occurred  through  the  Government's  focus  on  the  use  of  commercial  practices.  The 
1 986  Packard  Commission  Final  Report  concluded  that  the  use  of  commercial  practices 
in  the  DoD  acquisition  process  had  the  potential  for  saving  vast  amounts  of  Federal 
funds.  In  conjunction,  the  1986  Defense  Science  Board  stated  that  program  managers 
should  have  discretionary  authority  to  use  commercial  practices  and  products  when 
appropriate.  Commercial  practices  are  "techniques,  methods,  customs,  processes,  rules, 
guides,  and  standards,  normally  used  by  businesses,  but  either  applied  differently  or  not 
used  by  the  Federal  Government."  [Ref.  4:  p.  1-2]  One  such  practice  is  best  value.  In  the 
commercial  sector,  buyers  use  competition  to  their  advantage,  getting  the  maximum  value 
at  an  affordable  price.  The  objective  is  not  necessarily  lowest  price  or  maximum 
performance,  but  a  balance  between  the  two.  In  1989,  DoD  officially  recognized  the  best 
value  concept  in  a  study  entitled  Best  Value  Evaluation  Process.  The  Government  had  no 


set  definition  for  best  value  at  the  time  because  the  best  value  constantly  changed 
according  to  the  requirement  of  each  individual  purchase.  Best  value  was  generally 
accepted  to  mean  buying  on  other  than  a  price  only  basis.  [Ref.  4:  p.  1-1,  3-2] 
2.  Competition  In  Contracting  Act  (CICA)  Clarification 

A  1991  clarification  by  Congress  regarding  the  1984  Competition  in  Contracting 
Act  (CICA)  prompted  the  Government  to  focus  on  best  value.  CICA's  drive  for  free  and 
open  competition  was  sparked  by  the  assumption  that  competition  would  result  in  lower 
acquisition  costs  for  the  Government  as  well  as  expanding  contracting  opportunities  to 
every  level  of  business.  Initially  there  was  confusion  between  GAO  and  DoD  regarding 
the  legislation  because  many  in  DoD  improperly  interpreted  the  Act  as  defining  lowest 
cost  to  mean  lowest  price  rather  than  lowest  long-term  costs.  In  1991,  the  House  Armed 
Services  Committee  Report  for  the  Defense  Authorization  Act  issued  a  clarification 
stating  that  Congress  had  attempted  to  make  clear  that  lowest  overall  cost  was  not  limited 
to  price  and  price-related  factors.  Lowest  overall  cost  was  intended  to  be  the  outlay  or 
expenditure  that  the  Government  would  make  over  the  life  of  a  product.  The  committee 
went  on  to  state  that  cost  also  encompasses  technical  factors  such  as  quality,  design, 
technical  capability,  management  capability,  past  performance,  and  cost  discipline  to  the 
extent  these  factors  can  be  translated  into  a  monetary  context,  and  offerors  can  be  given  a 
clear  indication  in  the  solicitation  how  the  factors  would  be  quantified.  This  new 
interpretation  of  CICA  led  to  the  introduction  of  best  value.  [Ref.  5:  p.  59-60] 
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D.         EVOLUTION  OF  BEST  VALUE  CONTRACTING 
1.  Federal  Acquisition  Streamlining  Act 

The  Federal  Acquisition  Streamlining  Act  (FAS A)  of  1994  enacted  sweeping 
acquisition  reform  over  a  wide  spectrum  of  issues.  The  relationship  between  FASA  and 
best  value  was  indirect;  however,  it  is  important  to  note  that  this  Act  drove  many 
elements  of  the  FAR  Part  15  rewrite  and  its  specific  emphasis  on  best  value  contracting. 
A  key  concept  in  FASA  was  a  movement  of  Government  contracting  to  commercial 
methods.  This  requirement  of  commercial  contracting  methods  was  the  vital  element 
leading  to  the  Government's  future  use  of  best  value  acquisition. 

FASA  required  that  agencies  clearly  establish  the  relative  importance  assigned  to 
evaluation  factors  and  sub-factors.  FASA  specifically  addressed  the  quality  of  the 
product  or  services  to  be  provided  including  technical  capability,  management  capability, 
prior  experience,  and  past  performance  of  the  offeror.  Cost  was  carefully  addressed  as 
only  one  of  many  evaluation  factors  as  FASA  required  all  combined  evaluation  factors 
other  than  cost  or  price  be  disclosed  as  significantly  more  important,  approximately 
equal,  or  significantly  less  important  than  cost  or  price.  This  portion  of  the  Act  provided 
the  groundwork  for  the  tradeoff  process  which  is  essential  to  best  value  contracting. 
Through  its  recognition  of  other  evaluation  factors  besides  cost,  FASA  allowed 
Government  agencies  to  venture  into  the  spectrum  of  options  now  available  through  the 
use  of  tradeoffs.  [Ref.  6:  Sec.  1011] 
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2.  Clinton-Gore  Administration 

Best  value  contracting  received  high-level  endorsement  in  the  first  term  of  the 
Clinton-Gore  Administration.  Vice  President  Gore  in  the  Report  of  the  National 
Performance  Review:  Creating  a  Government  that  Works  Better  &  Costs  Less  recognized 
the  need  for  deregulation  in  the  acquisition  process.  The  report  maintained  that 
acquisition  regulations  should:  shift  from  rigid  rules  to  guiding  principles;  promote 
decision  making  at  the  lowest  possible  level;  end  unnecessary  regulatory  requirements; 
foster  competitiveness  and  commercial  practices;  and  shift  to  a  new  emphasis  on 
choosing  "best  value"  products.  In  early  1996,  the  FAR  Council  pursued  this  shift  to  best 
value  contracting  by  tasking  an  ad  hoc  interagency  committee  to  rewrite  FAR  Part  15. 
[Ref.  1:  p.  51224] 

3.  Federal  Acquisition  Regulation  (FAR)  Part  15  Rewrite 

FAR  Part  15  is  one  of  the  most  important  and  significant  parts  of  the  Federal 
Acquisition  Regulation  because  it  addresses  contracting  by  negotiation  and  competitive 
range  determination.  Within  the  Government,  the  majority  of  contracting  expenditures  are 
accomplished  using  Part  1 5  procedures.  The  Part  1 5  rewrite  is  a  normal  product  of  the 
continuous  improvement  process  employed  for  maintenance  of  the  FAR,  and  it  conforms 
with  the  general  reform  philosophy  espoused  by  the  Clinton-Gore  Administration.  FAR 
Part  15  was  rewritten  for  reasons  other  than  the  direction  of  the  Executive  Branch. 
Congress,  through  FASA  in  1994  and  the  Clinger-Cohen  Act  in  1996  dictated  so  many 
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fundamental  changes  in  Government  acquisition  that  the  rewrite  became  an  obvious 
necessity.  In  1995,  the  Department  of  Defense  (DoD)  conducted  a  survey  of  the  defense 
industry,  Military  Departments,  and  defense  agencies  to  ascertain  which  parts  of  the  FAR 
were  most  in  need  of  revision.  The  responses  indicated  a  general  consensus  that  Part  1 5 
was  one  of  the  parts  that  would  benefit  most  from  such  an  effort.  In  addition,  a  FAR 
Improvement  Study  conducted  by  the  General  Services  Administration  (GSA)  indicated 
that  subparts  15.6  (Source  Selection)  and  15.8  (Price  Negotiation)  were  the  most  difficult 
parts  of  the  FAR  to  use.  The  FAR  Part  15  rewrite  took  effect  on  10  October  1997.  [Ref. 
1:  p.  51225] 
E.         CURRENT  STATE  OF  BEST  VALUE 

The  current  state  of  best  value  contracting  from  the  perspective  of  the 
Government  is  contained  within  the  FAR  Part  1 5  rewrite.  In  sum,  best  value  is  the 
expected  outcome  of  any  acquisition  that  ensures  the  customer's  needs  are  met  in  the 
most  effective,  economical,  and  timely  manner.  It  is  the  result  of  the  combination  of:  the 
unique  circumstances  of  each  acquisition;  the  acquisition  strategy;  choice  of  contracting 
method;  and  the  award  decision.  Best  value  is  the  goal  of  sealed  bidding,  simplified 
acquisition,  commercial  item  acquisition,  negotiated  acquisition,  and  any  other 
specialized  acquisition  method  or  combination  of  methods.  Through  the  best  value 
continuum  the  Government  always  seeks  to  obtain  the  best  value  in  negotiated 
acquisitions  using  any  one  or  a  combination  of  source  selection  approaches,  and  that 
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acquisition  should  be  tailored  to  the  requirement.  At  one  end  of  this  continuum  is  the 
lowest  priced-technically  acceptable  strategy  and  at  the  other  end  is  a  process  by  which 
elements  of  a  proposed  solution  can  be  traded  off  against  each  other  to  determine  the 
solution  that  provides  the  Government  with  the  overall  best  value.  All  such  tradeoffs 
must  be  conducted  according  to  the  source  selection  factors  and  sub-factors  identified  in 
the  solicitation.  [Ref.  3:  p.  1] 
F.         CONCLUSION 

The  importance  of  best  value  cannot  be  overstated.  The  concept  has  been 
transformed  from  an  undefined,  nebulous  entity  to  the  mandated  goal  of  all  Government 
procurements.  Best  value  is  nothing  new  to  the  corporate  sector;  however,  the 
Government  is  a  relatively  new  participant  in  the  world  of  best  value.  Government 
contracting  officers  must  now  employ  this  form  of  contracting  without  the  benefit  of 
significant  experience  to  guide  the  way.  The  intent  of  this  thesis  is  to  counter  the 
Government  contracting  officers'  lack  of  best  value  experience  with  a  thorough  review 
and  analysis  of  recent  protests  of  best  value  contract  awards.  Chapter  III  will  review  all 
protests  of  contract  awards  under  best  value  criteria. 
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III.       OVERALL  REVIEW  OF  FISCAL  YEAR  1997  BEST  VALUE  PROTESTS 

A.  INTRODUCTION 

This  chapter  provides  an  overall  view  of  all  best  value  protests  for  which  the 
Comptroller  General  issued  decisions  in  1997.  This  chapter  is  beneficial  because  it  gives 
the  reader  a  firm  understanding  why  contractors  seeking  Government  business  protested 
best  value  contract  awards  in  1997.  First,  a  review  of  the  protest  process  is  conducted. 
Second,  the  methodology  used  to  define  a  best  value  protest  is  outlined.  Third,  the 
protest  sustainments  are  categorized.  Finally,  statistics  are  provided  for  each  sustainment 
category. 

B.  PROTEST  PROCESS 
1.  Purpose 

In  deciding  bid  protests,  GAO  considers  whether  Federal  agencies  have 
complied  with  statutes  and  regulations  controlling  Government  procurements.  As  stated 
in  Chapter  I,  Congress  exerts  its  right  to  influence  and  investigate  Government 
acquisition  through  the  system  of  award  protest.  Congress'  investigative  arm,  GAO 
provides  a  venue  for  contract  award  protest  and  renders  non-binding  but  influential 
decisions  regarding  source  selection.  [Ref.  2] 
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2.  The  Process 

The  bid  protest  process  begins  with  an  unsuccessful  offeror  filing  a  written  protest 
at  GAO.  Unless  the  protest  is  dismissed  due  to  procedural  or  substantive  defects  (e.g., 
the  protest  is  untimely  or  the  protest  fails  to  clearly  state  legally  sufficient  grounds  of 
protest),  the  contracting  agency  must  submit  an  agency  report  to  GAO  responding  to  the 
protest  and  a  copy  of  that  report  must  be  provided  to  the  protester.  The  protester  then 
may  file  written  comments  on  the  report. 

During  the  course  of  a  GAO  protest,  the  Comptroller  General  may  schedule 
informal  meetings  or  conferences  to  resolve  procedural  matters  and  obtain  information 
pertaining  to  the  disposition  of  the  protest.  GAO  may  also  find  that  a  hearing  must  be 
held  to  resolve  factual  and  legal  issues  raised  in  the  protest.  If  it  decides  to  hold  a 
hearing,  GAO  will  often  conduct  a  pre-hearing  conference  to  (1)  decide  the  issues  that 
will  be  considered  at  the  hearing,  (2)  identify  the  witnesses  who  will  testify  at  the 
hearing,  and  (3)  settle  procedural  questions.  After  the  hearing,  all  parties  may  submit 
written  comments. 

After  the  record  is  complete,  GAO  will  consider  the  facts  and  legal  issues  raised 
and  issue  a  decision.  A  copy  will  be  sent  to  all  parties  participating  in  the  protest.  GAO 
may  sustain  the  protest  (that  is,  find  that  the  agency  violated  a  procurement  statute  or 
regulation  and  prejudiced  the  protester),  in  which  case  GAO  will  recommend  appropriate 
corrective  action.  Conversely,  GAO  may  deny  the  protest  or  dismiss  the  protest  without 
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reviewing  the  matter.  GAO  issues  its  decisions  within  1 00  days  from  the  date  of  protest 

filing.  [Ref.  8] 

C.    METHODOLOGY  FOR  NARROWING  THE  BEST  VALUE  PROTESTS 

Several  assumptions  regarding  the  research  must  be  understood.    There  were  a 
total  of  146  decisions  rendered  by  GAO  on  best  value  contract  awards,  and  23  of  the 
protests  were  sustained.  Protests  are  initiated  by  contractors  for  many  reasons  such  as 
perceived  improprieties  in  the  solicitation,  competitive  range  determination,  or  evaluation 
of  the  various  proposals.  It  is  important  to  note  that  although  each  of  the  146  decisions 
were  about  a  best  value  contract,  some  of  the  protests  and  sustainments  had  nothing  to  do 
with  the  best  value  criteria  in  the  solicitation.  This  study  focuses  first  on  protests  that 
challenge  the  very  process  of  determining  the  best  value  offeror.  The  process  is  called  the 
tradeoff  process.  As  previously  discussed,  FAR  Part  15  defines  the  tradeoff  process  as 
part  of  the  best  value  continuum  where  dollars  are  traded  for  incremental  value  and  cost  is 
only  one  factor-not  the  controlling  factor.  The  tradeoff  process  is  appropriate  when  it 
may  be  in  the  best  interest  of  the  Government  to  consider  award  to  other  than  the  lowest 
priced  offeror  or  other  than  the  highest  technically  rated  offeror.  In  addition  to  the 
tradeoff  elements,  this  study  focuses  on  protests  that  challenged  Federal  agency  pre-  and 
post-award  changes.  These  protests  alleged  that  agency  actions  (changes)  invalidated  the 
best  value  source  selection.  For  the  purpose  of  this  study,  a  protest  of  a  best  value  award 
occurs  when  (1)  a  Federal  agency  evaluation  of  a  tradeoff  element  is  challenged  (e.g. 
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cost/price,  technical  merit,  past  performance,  labor  rates),  or  (2)  an  improper  agency  pre- 
or  post-award  change  occurs  that  compromises  an  otherwise  proper  best  value  award. 

D.  OVERALL  STATISTICS 

The  146  protests  and  23  sustainments  of  best  value  awards  were  narrowed  to  the 
108  protests  and  21  sustainments  that  met  the  researcher's  definition  of  a  best  value 
award  protest  (improper  agency  evaluation  of  a  tradeoff  element  or  improper  pre-  or  post- 
award  changes).  The  other  protests  were  simply  challenges  of  agency  evaluations  of 
peripheral  elements  that  were  in  a  best  value  solicitation  only  by  chance  (e.g.  competitive 
range  determinations,  constitutionality).    Of  these  108  protests,  87  were  denied  or 
dismissed,  and  21  were  sustained  yielding  a  sustainment  (success)  rate  of  19.44  percent. 
This  compares  to  an  overall  sustainment  rate  of  12  percent  for  all  protests  in  1997.  [Ref. 
9]    The  sustained  protests  will  be  categorized  and  discussed  for  the  remainder  of  the 
chapter.  The  Appendix  lists  all  146  protests  reviewed  during  the  research. 

E.  SUSTAINMENT  CATEGORIES 

The  2 1  sustainments  address  specific  allegations  regarding  Federal  agency 
evaluation  of  best  value  proposals.  Nearly  all  of  the  protests  cited  more  than  one  reason 
for  protest.  The  categorical  breakdown  outlined  below  represents  the  category  and 
success  rate  of  the  reasons  for  sustainment  of  the  21  best  value  protests. 
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1.  Improper  Agency  Evaluation  of  Past  Performance 

Past  performance  information  is  one  indicator  of  an  offeror's  ability  to  perform  a 
contract  successfully.  Relevant  past  performance  can  be  used  effectively  by  contracting 
officers  to  predict  the  capability  of  offerors.  Improper  Federal  agency  evaluation  of  past 
performance  was  a  sustaining  argument  in  six  (5.56  percent)  of  the  108  protests.  These 
six  sustainments  represent  28  percent  of  the  21  successful  best  value  protests. 

2.  Improper  Agency  Evaluation  of  Technical  Merit 

Evaluation  of  technical  merit  is  key  to  best  value  tradeoff  and  source  selection. 
Technical  merit  is  a  prime  indicator  of  quality.  Current  regulations  mandate  a  technical 
assessment  whenever  tradeoffs  are  performed.  [Ref.  1 :  15.305]  Improper  Federal  agency 
evaluation  of  technical  merit  was  a  sustaining  argument  in  seven  (6.48  percent)  of  the  108 
protests.  These  seven  sustainments  represent  33  percent  of  the  21  successful  best  value 
protests. 

3.  Improper  Agency  Evaluation  of  Cost/Price 

Cost/Price  is  clearly  one  of  the  most  important  elements  in  negotiated 
procurements.  It  is  also  a  central  tradeoff  element  in  best  value  source  selection. 
Improper  Federal  agency  evaluation  of  cost/price  was  a  sustaining  argument  in  six  (5.56 
percent)  of  the  108  protests.  These  six  sustainments  represent  28  percent  of  the  21 
sustainments. 
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4.  Improper  Agency  Evaluation  of  Labor  Qualifications 

Labor  qualifications  are  often  an  important  tradeoff  element  in  best  value  source 
selection.  Special  or  unique  qualifications  bring  value  to  an  offeror's  proposal  as  well  as 
cost.  Improper  Federal  agency  evaluation  of  labor  qualifications  was  a  sustaining 
argument  in  two  (1.56  percent)  of  the  108  protests.  These  two  sustainments  represent 
9.52  percent  of  the  21  successful  protests. 

5.  Improper  Agency  Pre-  or  Post-Award  Changes 

Improper  agency  pre-  or  post-award  changes  are  contracting  officer  actions  which 
undermine  and  invalidate  otherwise  proper  best  value  awards.  Improper  Federal  agency 
pre  or  post-award  change  was  a  sustaining  argument  in  three  (2.78  percent)  of  the  108 
protests.  These  six  sustainments  represent  14  percent  of  the  21  sustainments. 
Figures  3-1  and  3-2  outline  the  overall  and  sustainment  statistics. 
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F.         CONCLUSION 

This  chapter  displays  the  dominant  reasons  for  success  in  the  21  best  value 
sustainments  in  1997.  Improper  evaluation  of  tradeoff  elements  (past  performance, 
technical  merit,  cost/price,  and  labor  qualifications)  made  up  18  (85.71  percent)  of  the  21 
sustained  protests.  Improper  pre-  or  post-award  changes  were  sustaining  arguments  in 
only  3  of  the  21  sustainments;  however,  it  is  important  to  note  that  this  offense  was 
alleged  only  three  times  in  the  pool  of  108  best  value  protests.  In  other  words,  even 
though  improper  changes  were  alleged  infrequently,  they  were  successful  every  time. 
Chapter  IV  discusses  and  analyzes  the  six  protests  that  were  sustained  for  agency 
improprieties  in  past  performance  evaluation. 
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IV.       PAST  PERFORMANCE 
A.         INTRODUCTION 

Past  performance  information  is  one  indicator  of  an  offeror's  ability  to  perform  a 
contract  successfully.  [Ref.  1 :  15.305]     Improper  evaluation  of  past  performance  was  a 
sustaining  argument  in  six  (28  percent)  of  the  21  successful  best  value  protests  in  1997. 
This  chapter  details  all  six  protests  and  illustrates  common  elements  leading  to 
sustainment.    The  protests  in  which  improper  evaluation  of  past  performance  was  a 
sustaining  element  are  American  Combustion  Industries,  Inc.;  International  Business 
Systems,  Inc. ;  McHughf Calumet,  A  Joint  Venture;  Mechanical  Contractors,  S.A. ; 
NavCom  Defense  Elec  tonics,  Inc.;  and  ST  Aerospace  Engines  Pte.  Ltd.  The  six  protests 
are  divided  into  two  categories: 

1 .  Improper  evaluation  of  awardee  or  protester  past  performance; 

2.  Failure  to  conduct  meaningful  discussions  regarding  adverse  past  performance 
reports. 

One  of  the  protests  included  both  of  the  above  categories  as  successful  sustaining 
arguments.  This  chapter  discusses  current  and  past  regulations  regarding  past 
performance,  describes  the  six  sustained  protests,  and  provides  analyses  of  Federal 
agency  miscues  when  evaluating  past  performance. 
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B.         PAST  PERFORMANCE  AND  FAR  PART  15 

Past  performance  is,  by  statute,  a  mandatory  evaluation  element  of  all  negotiated 
source  selections.  It  is  also  a  key  tradeoff  element  in  best  value  contracting.  Before 
analyzing  best  value  sustainments  for  improper  evaluation  of  past  performance,  it  is 
important  to  understand  the  effect  of  the  FAR  Part  15  changes  that  were  published  on  30 
September  1997.  Past  performance  evaluation,  both  before  and  after  the  rewrite,  has 
several  basic  precepts.  First,  all  factors  and  significant  sub-factors  regarding  past 
performance  must  be  stated  in  the  solicitation.  In  addition,  the  offerors  shall  be  provided 
with  an  opportunity  to  identify  past  or  current  similar  contracts  as  well  as  an  opportunity 
to  comment  on  adverse  reports.  [Ref.  1 :  15.304]    While  the  rewrite  was  not  in  effect 
when  GAO  issued  most  of  the  1997  decisions,  the  planned  changes  were  well  known. 
With  respect  to  past  performance,  the  FAR  Part  1 5  changes  were  few.  One  significant 
difference  is  that  before  the  rewrite,  firms  lacking  relevant  past  performance  history 
received  a  Neutral  evaluation  for  past  performance.  [Ref.  13:  15.608]  The  rewrite 
modified  this  part  of  the  regulation  to  state  that  an  offeror  without  a  record  of  relevant 
past  performance  or  for  whom  information  on  past  performance  is  not  available,  may  not 
be  evaluated  favorably  or  unfavorably  on  past  performance.    In  addition,  the  rewrite 
states  that  the  Government  will  not  rely  on  adverse  past  performance  information  about 
which  the  contractor's  have  not  had  an  opportunity  to  comment.  [Ref.  1 :  15.305]  Even  if 
the  award  is  to  be  made  without  discussions,  offerors  must  be  given  the  opportunity  to 
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clarify  certain  aspects  of  their  proposals  (e.g.  relevance  of  an  offeror's  past  performance 
information  and  adverse  past  performance  information)  to  which  the  offeror  has  not 
previously  had  an  opportunity  to  respond.  [Ref.  1 :  15.306]  The  newest  exchange 
(communications)  allows  exchanges  prior  to  the  establishment  of  the  competitive  range  to 
assist  in  the  competitive  range  determination,  address  adverse  past  performance  reports, 
and  clear  ambiguities.    The  implication  for  the  contracting  officer  is  that  there  is  now  an 
avenue  outside  the  clarification/discussion  realm  that  allows  tailored  exchanges  with 
offeror(s)  about  adverse  past  performance  references. 

The  remainder  of  this  chapter  analyzes  the  six  protests  that  were  sustained  due  to 
improper  agency  evaluation  of  past  performance  in  1997. 
C.         IMPROPER  EVALUATION  OF  AWARDEE/PROTESTER  PAST 

PERFORMANCE 

1.  International  Business  Systems,  Inc. 

International  Business  Systems,  Inc.,  B-275554,  Comptroller  General  of  the 
United  States,  3  March  1997,  (IBSI)  was  sustained  due  to  agency  improprieties  in 
evaluating  past  performance.  IBSI  protested  the  Veteran's  Affairs  (VA)  award  of  a  fixed- 
priced  contract  for  a  replacement  telephone  system  for  the  VA  Medical  Center,  Wilkes- 
Barre,  Pennsylvania.  The  contract  competition  was  limited  to  participants  in  the  Small 
Business  Administration's  (SBA)  Section  8(a)  small,  disadvantaged  business  program. 
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The  VA  found  Dulles  Networking  Associates,  Inc.  (DNA)  to  be  the  offeror  whose 
proposal  was  determined  to  be  most  advantageous  to  the  Government. 

The  evaluation  scheme  involved  a  two-step  review.  First,  technical  proposals 
were  to  be  evaluated  on  a  pass/fail  basis.  Second,  the  technically  acceptable  offers  would 
be  reviewed  for  price  and  past  performance  which  were  weighted  equally  in  the  Request 
for  Proposals  (RFP).  Adjectival  ratings  were  to  be  assigned  to  each  offeror  for  past 
performance.  Three  proposals  including  IBSI  and  DNA  were  found  technically 
acceptable.  DNA  offered  the  lowest  price  ($  2.4  million)  and  was  issued  a  past 
performance  rating  of  Excellent.  IBSI  offered  a  price  of  ($  2.9  million)  and  was  issued  a 
past  performance  rating  of  Good. 

IBSI  argued  in  its  protest  that  DNA's  excellent  past  performance  rating  was 
unreasonable  because  DNA  had  no  direct  experience  furnishing  and  installing  telephone 
systems.  The  contracting  officer  (CO)  reevaluated  and  issued  a  Neutral  rating  to  DNA  in 
accordance  with  the  RFP.  In  reevaluating  IBSI's  proposal,  the  CO  identified  two 
references  directly  applicable  to  the  solicitation.  She  based  her  review  on  only  one 
reference  because  the  second  reference,  involving  the  installation  of  a  similar  telephone 
system  at  a  VA  Medical  Center  in  Massachusetts,  was  not  included  because  the 
individual  within  the  agency  responsible  for  completing  the  reference  form  failed  to  do 
so.  With  only  the  one  reference,  IBSI  was  again  given  a  Good  past  performance  rating. 
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The  CO  then  determined  that  a  Neutral  rating  (DNA)  was  essentially  equal  to  one  Good 
rating  (IBSI)  and  DNA's  lower  price  held  sway. 

IB  SI  argued  that  the  agency  was  required  to  consider  the  second  reference  (its 
installation  of  the  Massachusetts  Medical  Center  telephone  system).  VA  countered  and 
GAO  supported  the  assertion  that  there  was  no  legal  requirement  that  all  past 
performance  references  be  included  in  a  valid  review  of  past  performance.  The  winning 
factor  for  IBSI  was  in  the  fact  that  the  very  same  agency,  contracting  officer,  and  same 
services  were  involved  in  the  contract  award  and  administration  of  the  telephone 
installation  at  the  Massachusetts  VA  Medical  Center  (missing  second  reference).  In  fact, 
the  contracting  officer  stated  that  IBSI's  performance  had  been  "exemplary"  in  a  letter 
written  four  months  prior  to  the  award  decision  regarding  the  Pennsylvania  VA  Medical 
Center.  Under  these  circumstances,  GAO  held  that  the  agency  knew  of  the  past 
performance  and  unreasonably  failed  to  consider  IBSI's  positive  performance  in  the 
second  reference. 

Even  though  GAO  found  the  VA  evaluation  of  IBSI's  past  performance  to  be 
unreasonable,  competitive  prejudice  was  an  essential  element  that  had  to  be  established 
before  the  improper  evaluation  would  be  sustainable.  GAO  concluded  that  IBSI  was 
likely  prejudiced  by  the  agency's  failure  to  include  the  past  performance  in  the  second 
reference  (VA  Medical  Center  in  Massachusetts).  The  "exemplary"  performance  by  IBSI 
would  have  likely  garnered  at  least  a  rating  of  Good.  The  agency  would  then  have  to 


27 


weigh  IBSI's  two  Good  ratings  against  DNA's  one  Neutral  rating.  This  change  in  past 
performance  references  could  have  resulted  in  a  different  selection.  GAO  recommended 
a  reevaluation  of  the  proposals  including  IBSI's  two  references.  [Ref.  10] 

2.  Mechanical  Contractors,  S.  A. 

Like  IBSI,  Mechanical  Contractors,  S.  A,  B-277916,  Comptroller  General  of  the 
United  States,  27  October  1997,  (MECSA)  was  sustained  because  of  an  error  by  the 
Federal  agency.  MECSA  protested  the  Panama  Canal  Commission  (PCC)  award  of  a 
firm  fixed-price  contract  for  cleaning  and  painting  of  four  miter  gate  leaves  in  the  Panama 
Canal.  The  RFP  called  for  abrasive  blast  cleaning  and  exterior  painting  above  and  below 
the  water  line  using  hot-applied  coal  tar  enamel.  The  PCC  found  Formal  Management 
Systems,  Inc.  (FMS)  to  be  the  offeror  whose  proposal  offered  the  best  value  to  the 
Government. 

The  RFP  required  all  offerors  to  submit  a  technical  proposal  and  price  schedule. 
The  technical  proposals  were  to  be  evaluated  under  two  equally  weighted  factors: 
technical  approach  and  performance  capability.  Performance  capability  was  divided  into 
five  sub-factors  including  specialized  experience  and  past  performance.  The  total 
technical  evaluation  scores  were  to  be  weighted  equally  with  the  offerors'  firm  fixed- 
price  in  an  effort  to  determine  the  best  value  to  the  Government.    Three  proposals  were 
received  including  those  of  MECSA  and  FMS. 
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MECSA  proposed  the  lowest  price  of  $  2.5  million  while  FMS  submitted  a  price 
of  $  2.6  million.  The  key  difference  between  the  two  offerors  existed  in  the  performance 
capability  factor.  FMS  earned  81.55  points  in  performance  capability.  MECSA  earned 
66.2.  Past  performance  was  the  significant  difference  in  the  two  proposals  because  PCC 
awarded  the  contract  to  FMS  even  though  MECSA  proposed  a  lower  price.  The 
evaluation  record  showed  that  FMS  had  successfully  cleaned  and  coated  all  PCC  miter 
gates  for  the  previous  three  years.  MECSA  had  weaknesses  in  experience  and  past 
performances  as  the  agency  stated  they  had  very  little  coal  tar  enamel  work  over  the  same 
three  years. 

MECSA  argued  that  PCC's  evaluation  of  its  and  FMS's  past  performance  was 
unreasonable.  Offerors  were  to  list  similar  projects  over  the  past  three  years.  The 
Evaluation  Board  reviewed  1 7  contracts  and  concluded  that  three  ( 1 8  percent)  showed 
either  late  completion  or  liquidated  damages.  The  board  stated  that  MECSA  had  been 
late  on  "various  projects"  and  awarded  MECSA  only  a  50  percent  score  in  past 
performance.  GAO  found  the  reasonableness  of  the  score  in  doubt.    According  to  PCC's 
post-protest  submissions,  only  five  contracts  should  have  been  considered  similar  with 
one  (20  percent)  being  late.  It  is  important  to  note  that  although  the  performance  was 
cited  as  late,  MECSA  received  an  overall  performance  rating  of  Satisfactory  in  this 
similar  contract.  PCC  maintained  that  under  the  streamlined  number  of  contracts, 
MECSA' s  rate  of  late  performance  rose  from  1 8  to  20  percent  and  their  decision  would 


29 


have  remained  unchanged.  However,  GAO  held  that  with  only  one  suspect  similar 
contract,  the  key  downgrading  statement  regarding  late  performance  on  "various  projects" 
was  now  unsupported  and  unreasonable. 

In  contrast,  PCC's  evaluation  of  FMS's  past  performance  appeared  inconsistent 
with  the  evaluation  of  MECSA.  MECSA  pointed  out  that  FMS  had  serious,  documented 
safety  violations  (including  a  fatality)  on  previous  projects  that  had  no  affect  on  its  past 
performance  score.  PCC  stated  that  the  proposal  did  not  disclose  the  information,  and  the 
Evaluation  Board  did  not  weigh  these  facts.  However,  GAO  queried  the  board  regarding 
the  fatal  safety  violation  to  which  the  members  stated  that  they  did  have  knowledge  about 
the  incident  at  the  time  of  the  evaluation.  During  the  protest,  PCC  reevaluated  FMS's 
proposal,  taking  the  safety  violations  into  consideration,  and  still  maintained  that  FMS 
represented  best  value  for  the  Government. 

MECSA  also  argued  that  FMS  had  not  been  as  successful  when  cleaning  and 
painting  miter  gates  as  they  had  originally  maintained  in  their  proposal.  MECSA  pointed 
out  that  in  still  another  contract,  FMS  received  an  unsatisfactory  performance  rating  in  a 
safety  area.     PCC  maintained  that  since  FMS  had  garnered  a  satisfactory  performance 
rating  for  the  entire  contract,  it  was  reasonable  that  the  safety  problem  should  have  no 
affect  on  the  past  performance  rating.  GAO  disagreed  stating  that  this  approach  seemed 
inconsistent  with  the  downgrading  of  MECSA  for  late  performance  under  their  one  late 
contract  in  which  the  overall  performance  rating  had  been  satisfactory. 
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As  to  a  determination  of  prejudice,  GAO  stated  that  PCC's  reevaluation  of  the 
specialized  experience  of  FMS — taking  into  account  the  fatal  safety  violation — resulted 
in  only  a  small  difference  in  overall  adjectival  point  scores  (1 .2  points).  In  light  of  the 
protester's  lower  proposed  price,  GAO  determined  that  MECSA  would  have  had  a 
substantial  chance  of  receiving  the  award.  GAO  held  that  the  presence  of  the  multiple 
material  errors  in  the  evaluation  of  past  performance  and  technical  merit  (to  be  discussed 
in  a  later  chapter)  did  prejudice  MECSA.  Consequently,  GAO  recommended 
reevaluation.  [Ref.  11] 

3.  NavCom  Defense  Electronics,  Inc. 

NavCom  Defense  Electronics,  Inc.,  B-276163,  Comptroller  General  of  the  United 
States,  19  May  1997,  (NavCom)  was  similar  to  IBSI  and  MECSA  because  agency  miscues 
regarding  past  performance  resulted  in  sustainment  of  the  protest.  NavCom  protested  the 
U.  S.  Air  Force's  award  of  a  firm  fixed-price  contract  for  repair  of  over  104  line  items  in 
their  AN/ARN-1 1 8  Tactical  Air  Navigation  (TACAN)  systems.    The  contract 
competition  was  a  total  small  business  set  aside  for  one  base  year  and  four  one-year 
options.  The  Air  Force  found  Integrity  Air  Services,  Inc.  (IAS)  to  be  the  offeror  whose 
proposal  was  determined  to  be  most  advantageous  to  the  Government. 

The  TACAN  system  is  a  complex  collection  of  five  major  components:  (1)  a 
radio  receiver/transmitter;  (2)  a  radio  receiver  transmitter  control;  (3)  a  digital  to  analog 
adapter;  (4)  an  antenna;  and  (5)  a  mount.  The  system  contains  over  15,000  sub- 
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components.  The  RFP  stated  two  evaluation  factors — performance  and  price — were  of 
equal  importance  and  best  value  would  be  determined  through  a  performance/price 
tradeoff.  Eight  proposals  were  received.  Under  price,  IAS  ranked  first  with  the  lowest 
proposed  price  and  NavCom  ranked  fourth.  The  RFP  required  offerors  to  forward 
information  regarding  current  and  past  performance  on  "the  same  or  similar"  efforts.  Past 
performance  was  evaluated  as  Low,  Medium,  or  High  risk.  All  offerors  earned  risk  scores 
of  Low  regarding  past  performance  as  the  Air  Force  determined  that  all  firms  had  the 
desired  "similar  or  same"  experience  with  the  TACAN  system.  With  equal  past 
performance  ratings,  the  lower  priced  proposal  by  IAS  won  the  award. 

NavCom  protested  asserting  that  IAS  did  not  have  "same  or  similar"  past 
performance  and  should  have  received  a  risk  rating  of  other  than  Low.  IAS  past  efforts 
consisted  of  only  four  contracts  to  repair  radio  power  supplies,  and  none  of  the  contracts 
were  valued  at  greater  than  $  1  million.  Power  supplies  were  not  a  line  item  for  repair  in 
the  RFP.  However,  the  solicitation  did  state  that  the  contractor  must  possess  knowledge 
of  direct  and  alternating  current  circuitry,  analog  and  digital  signals,  timing  networks, 
high  frequency  networks  and  electronics,  pulse  width  decoding,  synchro  technology, 
built-in  test  circuitry,  operation  of  systems  bearing  and  distance  measurement  signals, 
radio  receiving  technology,  and  pulsed  transmission  technology.  [Ref.  12] 

In  contrast,  NavCom  was  the  only  offeror  with  satisfactory,  direct  experience  with 
the  TACAN  system.  NavCom' s  previous  TACAN  contract  was  valued  at  over  $  8 
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million.  The  Air  Force  acknowledged  that  the  performance  risk  assessment  team  was 
fully  aware  of  NavCom's  past  experience;  moreover,  they  were  equally  aware  that  the 
remaining  offerors  had  no  actual  TACAN  experience.  Still,  all  offerors  received 
performance  risk  ratings  of  Low.  GAO  stated  that  the  record  lacked  any  basis  upon 
which  the  Air  Force  could  reasonably  conclude  that  IAS's  repair  of  power  supplies  was 
"similar  or  same." 

The  Air  Force  stated  that  consideration  was  made  to  the  fact  that  this  was  a  1 00 
percent  small  business  set-aside.  They  maintained  that  very  few  small  businesses  have 
experience  with  contracts  the  size  of  the  TACAN  effort.  Consequently,  it  became  a 
question  of  policy  whether  to  award  contracts  to  small  business  to  expand  the  contracting 
base.  GAO  stated  the  goal  of  contracting  base  expansion  does  not  allow  the  agency  to 
disregard  the  terms  of  the  RFP.  GAO  concluded  by  stating  that  only  NavCom  had  the 
"same"  experience  with  the  TACAN,  and  the  agency  provided  no  evidence  in  the  record 
establishing  that  IAS  had  "similar"  experience.  GAO  could  not  conclude  that  the  award 
to  IAS  was  reasonable,  and  they  found  that  NavCom  had  been  prejudiced.  GAO 
sustained  the  protest  and  recommended  reevaluation.  [Ref.  12] 
D.         FAILURE  TO  CONDUCT  MEANINGFUL  DISCUSSIONS  REGARDING 

ADVERSE  PAST  PERFORMANCE  REPORTS 

Three  of  the  six  protests  in  which  improper  evaluation  past  performance  was  a 
sustaining  argument  hinged  on  agency  failure  to  conduct  meaningful  discussions 
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regarding  adverse  past  performance  reports.  It  is  important  to  note  that  the  mere  absence 
of  required  meaningful  discussions  about  adverse  past  performance  reports,  when 
required,  is  by  itself  a  sustainable  argument. 

1.  American  Combustion  Industries,  Inc. 

American  Combustion  Industries,  Inc.,  B-275057.2,  Comptroller  General  of  the 
United  States,  5  March  1997,  (ACI)  was  a  protest  in  which  an  agency  failure  to  hold 
discussions  resulted  in  a  protest  sustainment.  ACI  protested  the  Department  of 
Commerce's  National  Institute  of  Standards  and  Technology's  (NIST)  award  of  a  fixed- 
price  construction  contract  for  two  boilers  and  boiler  housing.    The  contract  was  for  all 
labor,  equipment,  and  material  for  the  supply  and  installation  of  two  82,000  pounds  per 
hour  boilers  and  construction  of  an  addition  to  an  existing  building  to  house  the  boilers. 
NIST  determined  that  Green  Contracting  Company,  Inc.  (GCC)  offered  a  proposal 
representing  the  best  value  to  the  Government. 

The  RFP  established  six  evaluation  factors,  three  of  which  involved  past 
performance.  Past  performance  factors  carried  a  total  assigned  weight  of  70  percent. 

1 .  Past  Performance  on  Building  Construction  (30  percent) 

2.  Past  Performance  on  Phased  Refurbishing  (20  percent) 

3.  Past  Performance  of  Personnel  (20  percent) 

GAO  noted  improprieties  in  the  evaluation  of  the  past  performance  of  personnel,  and  this 
will  be  discussed  in  a  later  chapter  involving  improper  evaluation  of  labor  qualifications. 
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The  clear  past  performance  impropriety  in  this  protest  occurred  in  the  agency's  attempt  to 
evaluate  the  past  performance  on  building  construction.  The  RFP  required  offerors  to 
provide  references  on  past  construction  projects.  In  accordance  with  the  RFP,  ACI 
identified  two  construction  projects.  One  was  with  the  National  Aeronautics  and  Space 
Administration  (NASA)  and  the  other  with  James  Madison  University  (JMU)  in  Virginia. 

NASA  advised  the  agency  that  ACI  was  at  least  100  days  late  in  beginning  work 
and  that  they  had  problems  with  the  firm's  project  manager.  JMU  reported  that  the 
protester  was  five  months  late  due  to  slow  delivery  of  a  boiler,  in  part,  because  of  ACI. 
The  JMU  reference  was  otherwise  positive.  The  agency  held  discussions  with  ACI 
regarding  NASA  but  not  JMU.  The  agency  stated  that  FAR  15.610(c)(6)  anticipated  an 
eventual  implementation  of  a  Government  past  performance  reporting  network.  [Ref.  13] 
NIST  further  stated  that  the  FAR  requirement  to  discuss  adverse  past  performance  was 
not  in  effect  until  the  past  performance  reporting  network  was  implemented.  GAO 
rebuked  this  notion  stating  that  the  FAR  plainly  mandated  an  opportunity  for  offerors  to 
reply  to  adverse  past  performance  information.  [Ref.  14] 

The  protester  also  alleged  improper  evaluation  of  past  performance  regarding  their 
project  manager.  The  solicitation  required  the  offerors  to  identify  certain  key  personnel 
who  would  be  involved  in  the  boiler/boiler  housing  work.  ACI  identified  a  project 
manager  who  was  regarded  highly  by  the  agency.  However,  the  proposed  project 
manager  did  not  appear  (as  requested  by  the  agency)  at  discussions  conducted  by  NIST. 
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Since  the  proposed  project  manager  was  unavailable  for  the  discussions,  ACI  sent  its 
president  and  vice-president  in  his  place.  Although  the  transcript  of  the  discussions 
showed  different  understandings  about  what  was  said  regarding  the  project  manager's 
absence,  the  agency  was  concerned  that  the  vice-president  in  attendance  would  be 
substituted  for  the  project  manager  for  the  initial  months  of  contract  performance.  The 
evaluation  record  showed  several  key  facts: 

1 .  The  vice-president  who  was  assumed  to  be  a  replacement  for  the  project 
manager  was  viewed  as  a  poor  choice  because  of  negative  references. 

2.  NIST  did  not  discuss  its  concerns  during  the  discussions. 

3.  ACI  did  not  amend  its  proposal  to  show  a  replacement  of  the  proposed  project 
manager  with  the  vice-president. 

4.  NIST  deducted  points  from  ACI  under  the  past  performance  of  personnel 
category. 

GAO  held  that  the  agency's  failure  to  advise  ACI  of  their  project  manager 
assumption  unreasonably  denied  the  protester  a  chance  to  resolve  the  question  about 
personnel  availability.  Given  that  a  major  deduction  from  ACI's  score  was  due  to  the 
perceived  substitution  with  the  vice-president,  GAO  held  that  the  protester  had  been 
prejudiced.  GAO  also  held  that  the  ACI  was  prejudiced  by  the  lack  of  discussions 
regarding  JMU.    GAO  recommended  discussions,  revised  BAFOs,  and  reevaluation. 
[Ref.  14] 
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2.  McHugh/Calumet,  A  Joint  Venture 

Mc Hugh/Calumet,  A  Joint  Venture,  B-276472,  Comptroller  General  of  the  United 
States,  23  June  1997,  {McHugh)  like  AC  I,  was  a  protest  that  was  sustained  because 
discussions  were  not  held  about  poor  past  performance  references.  McHugh/Calumet 
protested  the  General  Services  Administration  (GS  A)  award  of  a  fixed-priced  contract  for 
construction  of  a  new  Federal  courthouse  in  Hammond,  Indiana.  GSA  found  Huber, 
Hunt,  &  Nichols,  Inc.  (HHN)  to  be  the  offeror  whose  proposal  was  determined  to  be  most 
advantageous  to  the  Government. 

The  RFP  called  for  the  construction  of  a  270,000-square  foot,  four-story, 
limestone  courthouse  containing  seven  courtrooms,  Federal  office  space,  a  cafeteria,  and 
a  firing  range.  The  evaluation  elements  were  price  and  technical  factors.  Each  element 
was  weighted  equally.  The  technical  factors  consisted  of  (1)  quality  control — 40 
percent;  (2)  past  performance  on  similar  projects — 35  percent;  and  (3)  key  personnel 
qualifications — 25  percent.  Regarding  price,  McHugh' s  proposal  of  $  49  million  was 
lower  than  HHN's  by  over  a  million  dollars.  GSA  felt  that  McHugh' s  past  performance 
on  similar  projects  was  lacking  compared  to  HHN's.  The  RFP  required  one-page 
descriptions  and  references  for  projects  of  similar  scope  completed  in  the  last  five  years. 
HHN  had  satisfactorily  worked  on  three  courthouses  with  "directly  comparable"  or  "very 
similar"  architectural  (wood  paneled  courtrooms)  finishes.  McHugh  had  constructed  no 
courthouses  during  the  five-year  period,  and  the  descriptions  that  they  submitted  were 
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seen  to  be  only  "similar"  by  GSA.  McHugh  did  list  two  courthouses  from  before  the 
five-year  period  which  GSA  found  to  be  "not  nearly  similar"  because  they  did  not  have 
the  same  level  of  quality  finishes. 

HHN's  overall  advantage  was  based  not  only  on  greater  similarity  in  projects  but 
on  better  past  working  relationships.  HHN  received  scores  of  Above  Average  for  working 
relationships  on  two  submitted  references  and  a  score  of  Outstanding  on  a  third. 
McHugh  had  two  scores  of  Outstanding  and  one  Below  Average.  Discussions  were  not 
held  regarding  the  below  average  score  because  it  was  an  internal  agency  reference,  and 
GSA  maintained  that  the  FAR  15.610  (c)(6)  requirement  for  discussions  applied  only  to 
third  party  references.  GSA  also  stated  that  internal  agency  references  were  clear  and  not 
subject  to  interpretation,  so  discussions  would  be  pointless.  GAO  found  no  basis  why 
FAR  15.610  (c)(6)  would  not  apply  in  this  case  by  stating  that  nothing  in  the  FAR  limits 
its  application  to  a  third  party.  [Ref.  13]  GAO  also  found  that  McHugh  had  been 
prejudiced  because  discussions  would  have  furnished  McHugh  the  opportunity  to  contend 
that  the  poor  reference  was  a  personality  conflict  and  not  performance  related.  In  light  of 
the  lower  price,  close  competition,  and  scores  of  Outstanding  on  the  other  references, 
GAO  held  that  McHugh  had  a  reasonable  chance  of  obtaining  award  if  they  were  afforded 
the  opportunity  to  refute  the  poor  internal  agency  reference.  GAO  recommended 
discussions,  revised  BAFOs,  and  reevaluation.  [Ref.  15] 
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3.  ST  Aerospace  Engines  Pte.  Ltd. 

ST  Aerospace  Engines  Pte.  Ltd.,  B-275725,  Comptroller  General  of  the  United 
States,  19  March  1997,  {ST A  Engines)  is  a  protest  involving  the  poor  past  performance  of 
an  affiliate  as  well  as  a  failure  by  a  Federal  agency  to  hold  discussions.  STA  Engines 
protested  the  Department  of  Transportation's  U.  S.  Coast  Guard  (USCG)  award  of  a 
contract  for  the  overhaul  and  repair  of  helicopter  reduction  gearboxes  and  torquemeters 
used  in  the  T-56  engines  of  C- 130  aircraft.  USCG  found  Standard  Aero  Ltd.  (Standard)  to 
be  the  offeror  whose  proposal  was  determined  to  be  the  best  value  for  the  Government. 

The  RFP  stated  that  the  responsible  offeror  with  the  best  price/technical  merit 
combination  would  be  found  to  be  the  best  value  offeror  and  obtain  the  award.  Technical 
merit  consisted  of  (descending  order  of  importance)  past  performance,  certification, 
industrial  capability/capacity,  warranty,  and  engineering  and  support  capability.  The 
evaluators  stated  that  STA  Engines  was  downgraded  primarily  due  to  concerns  about  the 
company's  past  record  of  late  delivery.  USCG  pointed  out  that  the  protester's  past 
performance  was  for  the  most  part  good  except  for  significantly  late  deliveries  of 
overhauled  ship  propellers  under  a  separate  USCG  contract. 

STA  Engines  maintained  that  the  negative  past  performance  rating  was 
unreasonable  because  they  had  not  participated  in  the  contract  in  question.  STA  Engines 
asserted  that  it  was  their  affiliate  ST  Aerospace  Systems  (STA  Systems)  that  had 
performed  poorly.  STA  Engines  further  maintained  that  had  they  the  opportunity  to 
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discuss  the  poor  report,  they  could  have  vindicated  themselves  and  won  the  award. 
USCG  countered  stating  that  STA  Engines  proposal  had  held  the  two  as  affiliated  and 
even  forwarded  the  propeller  contract  as  a  reference.  GAO  stated  that  affiliation  alone 
was  not  a  deciding  factor  regarding  STA  Engines.  GAO  pointed  out  that  the  relationship 
between  the  two  was  key.    The  critical  question  was  whether  the  workforce, 
management,  facilities,  or  other  resources  of  one  company  may  affect  the  contract 
performance  of  the  affiliate. 

GAO  held  that  the  agency  had  provided  no  evidence  that  the  past  performance  of 
STA  Systems  was  of  any  relevance  to  the  potential  contract  performance  of  STA 
Engines.  Furthermore,  the  agency  was  required  to  raise  the  issue  of  the  affiliate's 
performance  and  relationship  with  STA  Engines  during  discussions.  Specific  weaknesses 
should  have  been  pointed  out.  Had  the  discussions  been  held,  STA  Engines  would  have 
been  able  to  challenge  any  negative  past  performance  of  any  of  the  other  units  in  the 
parent  company~ST  Aerospace  Group.  GAO  held  STA  Engines  was  prejudiced  by  both 
the  improper  attribution  of  an  affiliate's  negative  past  performance  and  the  failure  of 
USCG  to  hold  meaningful  discussions.  GAO  recommended  discussions,  revised  BAFOs, 
and  reevaluation.  [Ref.  1 6] 
E.         ANALYSIS  AND  CONCLUSION 

The  purpose  of  this  analysis  is  to  give  the  contracting  officer  an  indication  of  the 
common  pitfalls  when  evaluating  past  performance  in  best  value  contracts. 
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1.  Follow  the  Solicitation  in  a  Consistent  Manner 

It  is  clear  in  several  protests  that  failure  to  follow  the  solicitation's  evaluation 
scheme  led  to  the  improper  evaluation  of  past  performance  and  sustainment.  NavCom  is 
a  good  example.  In  this  protest,  we  see  that  an  RFP  placing  emphasis  on  "same  or 
similar"  past  performance  must  be  followed.  The  RFP  was  clear  in  this  case  where  past 
TACAN  experience  was  required,  yet  the  contracting  officer  seemed  to  arbitrarily  judge 
every  offerors'  past  performance  as  equal  (low  risk).  GAO  held  that  the  agency 
evaluation  was  not  consistent  with  the  solicitation.  This  decision  stemmed  from  the 
fundamental  requirement  for  agency  evaluations  to  be  reasonable  and  consistent  with 
stated  evaluation  criteria.  GAO  cited  a  previous  decision  in  support  of  their  finding 
(Ogden  Support  Services,  Inc.,  B-270012.2,  19  March  1997). 

NavCom  also  shows  us  that  circumventing  the  RFP  for  unstated  policy  goals  (e.g. 
increasing  the  industrial  base)  is  not  supported  by  the  Comptroller  General.  In  this 
protest  such  a  goal  existed,  but  the  RFP  contradicted  the  goal  by  placing  a  high  premium 
on  "same  or  similar"  past  performance.  The  Federal  agency  could  not  both  follow  the 
RFP  and  achieve  their  goal  of  increased  industrial  capacity.    GAO  held  that  the  agency 
evaluation  was  unreasonable.  Their  holding  stemmed  from  the  Comptroller  General's 
assertion  that  the  agency  cannot  disregard  the  terms  of  the  RFP  in  evaluating  an  offeror's 
proposal — even  though  the  agency  has  an  outlying  goal  of  industrial  base  expansion.  The 
implication  from  NavCom  is  twofold.    First,  contracting  officer  judgment  is  often 
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supported  by  GAO,  but  the  contracting  officer  must  be  reasonable,  consistent,  and  in 
accordance  with  the  RFP.  Second,  peripheral  agency  goals  do  not  absolve  the  contracting 
officer  from  the  requirement  to  adhere  to  the  solicitation.  If  an  increased  industrial  base  is 
the  priority,  then  the  RFP  should  allow  for  inexperience  and  place  less  the  emphasis  on 
past  performance.  However,  there  is  a  risk  of  poor  or  non-performance  with  this  strategy. 
CICA  certified  Congress'  intent  to  maximize  competition,  but  the  contracting  officer  is 
bound  to  evaluate  the  proposals  reasonably  and  consistently  in  accordance  with  the  RFP. 
2.  Make  Use  of  Relevant,  Known  Facts  About  Offerors 

MECSA  demonstrates  problems  with  agency  use  of  relevant  data.  The  awardee  in 
this  protest  had  several  previous  safety  violations  that  were  known  to  the  evaluation 
board.  Still,  the  agency  chose  to  ignore  the  violations  and  did  not  downgrade  the  offeror. 
The  key  here  is  relevance.  GAO  found  that  in  the  dangerous  activity  of  blasting, 
preserving,  and  painting  Panama  Canal  miter  gates  above  and  below  the  water  line,  safety 
was  a  relevant  issue.  The  evaluation  board  ignored  these  facts  in  its  past  performance 
evaluation,  and  it  was  a  sustaining  argument  for  the  protester.  GAO  held  that  the  Federal 
agency  was  not  consistent  in  its  evaluation.  The  holding  was  rooted  in  a  previous  decision 
{Boeing  Sikorsky  Aircraft  Support;  B-277263.2,  B-277263.3;  29  September  7997)  that 
stated  offerors  must  receive  the  fair  and  considered  judgment  of  the  agency.  The 
implication  for  the  contracting  officer  is  that  inconsistency  in  application  of  evaluation 
criteria  is  an  obvious  but  fatal  flaw  in  past  performance  evaluation. 
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/2?SY  reinforces  this  theme.  This  protest  exposes  the  disregarding  of  known, 
positive  information  by  an  agency  that  could  have  helped  the  protester.  Not  only  had  the 
Veteran's  Administration  (VA)  improperly  given  the  awardee  a  good  past  performance 
rating  in  the  light  of  no  direct  experience,  but  an  agency  employee  forgot  to  fill  out  a 
reference  form  and  reduced  the  protester's  pool  of  positive  references  from  two  to  one.  It 
is  important  to  note  that  in  IBSI,  the  mistake  by  the  agency  was  not  the  cornerstone  to 
sustainment.  There  is  no  legal  requirement  that  all  past  performance  references  be 
included  in  a  valid  evaluation.  Rather,  the  fact  that  the  missing  second  reference  involved 
the  same  services,  Federal  agency,  and  contracting  officer  caused  the  Comptroller 
General  to  find  that  the  previous  "exemplary"  performance  of  the  protester  was  a  known 
fact  to  the  VA.  GAO  held  that  it  was  unreasonable  for  the  Federal  agency  to  ignore  this 
known  and  relevant  past  performance.  The  Comptroller  General  supported  its  decision 
by  citing  a  previous  protest  sustainment  (Marine  Diesel,  P  hilly  ship;  B-232619,  B- 
232619.2;  27  January  1989)  where  the  Navy  chose  not  to  consider  unsatisfactory  past 
performance  of  an  awardee  involving  similar  services  and  the  same  command  because  the 
awardee  did  not  include  the  controversial  contract  on  its  list  of  references.  The 
implication  for  the  contracting  officer  is  that  an  agency  cannot  simply  choose  to  ignore 
relevant  information.    The  failure  to  utilize  relevant  data  could  result  sustainment. 

Relevance  is  a  double-edged  sword.  In  STA  Engines,  the  USCG  applied  irrelevant 
past  performance  data  about  an  affiliate  and  unreasonably  penalized  the  protester.  This  is 
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not  to  say  that  the  performance  of  affiliates  is  always  irrelevant.  GAO  stated  that  the 
relationship  determines  the  relevance.  If  the  resources  and  actions  of  one  affiliate  can 
affect  the  potential  contact  performance  of  another,  then  the  affiliate's  past 
performance — positive  or  negative — is  relevant.  GAO  held  that  the  agency  had  not 
demonstrated  that  the  past  performance  of  the  protester's  affiliate  was  relevant.  The 
holding  stems  from  a  previous  protest  {Contract  Services  Co.,  Inc.,  B-246604.2,  11  June 
1992)  where  GAO  stated  that  it  would  be  inappropriate  to  consider  the  affiliate's  record 
where  that  record  did  not  bear  on  the  likelihood  of  successful  performance  by  the 
protester.  The  implication  for  the  contracting  officer  is  that  an  affiliate's  past 
performance  must  be  carefully  screened  for  relevance.  Relevant  past  performance  data 
should  be  used,  and  irrelevant  data  should  be  disregarded. 

3.  Hold  Discussions  Regarding  Negative  Past  Performance 

Discussions  regarding  poor  past  performance  references  must  be  held  to  give  the 
offerors  an  opportunity  to  explain  the  situation.  This  was  a  frequent,  sustaining  error  by 
Federal  agencies  in  three  protests.  ACI  shows  that  although  the  FAR  is  and  was  clear  in 
this  matter,  mistakes  can  be  made.    The  agency  (NIST)  held  discussions  with  the 
protester  regarding  a  poor  reference  from  NASA  but  elected  not  to  do  so  regarding  James 
Madison  University  (JMU).  The  reason  was  the  agency's  perceived  delay  in  the  FAR's 
requirement  to  discuss  poor  references  until  an  anticipated  past  performance  reporting 
network  was  implemented.  GAO  held  that  discussions  were  required.  This  finding  was 
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rooted  in  the  fact  that  the  FAR  (15.610)  was  clear,  and  the  absence  of  the  anticipated 
reporting  network  in  no  way  relieved  a  Federal  agency  from  its  duty  to  inform  an  offeror 
of  a  poor  past  performance  report.  The  implication  for  the  contracting  officer  is  that  poor 
past  performance  reports  must  be  addressed  to  the  offeror  to  provide  an  opportunity  for 
comment. 

McHugh  demonstrates  that  an  assumption  about  the  source  of  a  reference  can  be 
troublesome.  GSA  believed  the  requirement  to  discuss  a  negative  past  performance 
existed  only  for  third  party  references.  Internal  agency  references  were  assumed  to  be 
clear  and  not  subject  to  interpretation;  therefore,  discussions  were  pointless  and  not 
required.  However,  the  Comptroller  General  found  no  basis  why  the  FAR  requirement 
for  discussions  would  not  apply  in  this  case  and  sustained  the  protest.  This  holding  stated 
that  nothing  in  the  FAR  limited  the  application  of  the  requirement  for  discussions  to  third 
party  references.  Contracting  officers  should  be  wary  of  past  performance  information 
that  they  consider  indisputable  (e.g.  intra-agency  references).  The  source  of  a  poor  report 
does  not  affect  the  contracting  officer's  requirement  to  provide  the  offeror  with  a  chance 
to  address  the  situation. 

As  previously  stated,  STA  Engines  demonstrates  that  an  affiliate's  past 
performance  may  not  necessarily  be  relevant  to  a  sister  company's  ability  to  perform  the 
terms  of  a  contract.  STA  Engines  also  shows  us  that  the  holding  of  discussions  may  have 
precluded  the  improper  use  of  the  affiliate's  past  performance.  STA  Engine's  had  not 
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been  given  the  opportunity  to  respond  to  the  agency's  contention  that  their  affiliate's 
performance  was  relevant  to  their  ability  to  perform.  If  discussions  had  been  held,  the 
problem  may  have  been  averted.  In  any  case,  GAO  held  that  the  lack  of  discussions  was 
a  sustaining  argument  for  the  protester.  GAO  bolstered  its  holding  by  citing  a  previous 
protest  {Alliant  Techsy stems,  Inc.,  Olin  Corporation;  B-260215.4,  B-260215-5;  4  August 
1995)  that  stated  for  discussions  to  be  meaningful,  an  agency  must  point  out  significant 
weaknesses  in  a  proposal  that  would  prevent  the  offeror  from  having  a  reasonable  chance 
for  award.  The  contracting  officer  must  realize  that  discussions  assist  the  agency  by 
identifying  potential  improprieties  in  the  evaluation  before  award  is  made.  Had 
discussions  been  held,  objections  by  the  protester  could  have  caused  the  agency  to  review 
the  situation  further  and  make  proper  evaluation  decisions. 

A  final  observation  can  be  drawn  regarding  required  discussions.  In  two  of  the 
three  cases,  the  Federal  agencies  maintained  that  the  protester  knew  about  the  poor  past 
performance  and  that  their  knowledge  released  the  agencies  from  their  obligation  to  hold 
discussions.  In  both  cases,  GAO  held  that  knowledge  of  negative  past  performance  was 
not  the  issue.  GAO  indicated  that  contracting  officers  must  inform  an  offeror  of  the 
impact  of  the  reference.  Federal  agencies  must  state  if  an  offeror's  past  performance 
detracts  from  their  ability  to  win  the  award.  Discussions  (communications)  must  be  held. 

Contracting  officers  should  take  many  valuable  lessons  from  this  chapter. 
Consistent,  relevant,  evaluation  of  a  proposal  in  accordance  with  the  solicitation  is 
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crucial.  Underlying  policy  goals  do  not  allow  a  Federal  agency  to  disregard  the  terms  of 
a  RFP.  All  known  facts  within  the  confines  of  the  evaluation  criteria  must  be  used  when 
considering  past  performance.  Irrelevant  facts  must  be  excluded.  Finally,  discussions 
must  be  held  to  afford  each  offeror  an  opportunity  to  refute  a  negative  past  performance 
report  even  if  the  offeror  is  aware  of  the  poor  reference. 
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V.  TECHNICAL  MERIT 

A.         INTRODUCTION 

Evaluation  of  technical  merit  is  a  cornerstone  of  best  value  tradeoff  and  source 
selection.  Current  regulation  dictates  that  when  tradeoffs  are  performed,  the  source 
selection  records  shall  include  (1)  an  assessment  of  each  offeror's  ability  to  accomplish 
the  technical  requirements  and  (2)  a  summary,  matrix,  or  quantitative  ranking  along  with 
appropriate  supporting  narrative  of  each  technical  proposal  using  the  evaluation  factors. 
[Ref.  1 :  15.305(a)(3)]  This  chapter  focuses  on  Federal  agency  technical  assessment, 
ranking,  and  supporting  narratives  when  evaluating  proposals  in  an  effort  to  make  best 
value  awards.  The  seven  protests  in  which  improper  evaluation  of  technical  merit  was  a 
sustaining  argument  are  Cygnus  Corporation;  HG  Properties  A,  L.  P.;  J.  A.  Jones 
Management  Services,  Inc.;  JW Associates,  Inc.;  International  Data  Systems,  Inc.; 
Mechanical  Contractors,  S.  A. ;  and  Technology  Services  International,  Inc.  The  seven 
protests  are  divided  into  two  categories: 

1 .  Improper  evaluation  of  technical  merit; 

2.  Improper  evaluation  of  technical  merit  combined  with  inadequate  supporting 
documentation. 
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This  chapter  discusses  the  differences  in  governing  regulations  as  a  result  of  the  FAR  Part 
1 5  rewrite,  outlines  the  seven  sustained  protests,  and  provides  analysis  and  conclusions 
regarding  the  reasons  for  sustainment. 
B.         TECHNICAL  MERIT  AND  FAR  PART  15 

Only  a  few  of  the  FAR  requirements  regarding  technical  evaluation  changed  due 
to  the  Part  1 5  rewrite.  One  key  difference  is  the  requirement  for  appropriate  supporting 
narrative  to  accompany  the  previously  required  summaries  (strengths,  weaknesses,  and 
basis  for  award),  matrices,  and  quantitative  rankings  when  making  a  best  value  source 
selection.  The  FAR  Part  1 5  rewrite  states  that  the  rationale  for  tradeoffs  must  be 
documented  in  the  contract  file.  While  the  stated  requirement  for  tradeoff  rationale  was 
not  in  force  at  the  time  of  these  protests,  it  appears  that  GAO  may  have  intended  to  send  a 
message  regarding  the  importance  of  these  narratives  when  they  firmly  stressed  the  need 
for  supporting  information  in  three  of  the  protest  decisions.  [Ref.  1:  15.101-1] 

In  a  separate  issue,  the  Part  1 5  rewrite  encouraged  open  exchanges  (clarifications, 
negotiations/discussions,  and  communications)  in  an  effort  to  achieve  best  value.  The 
term  communications  was  added  to  allow  better  dialog  leading  the  establishment  of 
competitive  ranges.  Clarifications  and  discussions  remained  essentially  unchanged  in 
their  definition  and  application.  The  distinction  between  clarifications  and  discussions  is 
a  key  element  in  one  of  the  protests  in  this  chapter.  It  is  important  to  note  that 
clarifications  are  limited  exchanges  of  information  to  clarify  ambiguities  when  award 
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without  discussions  is  contemplated.  Discussions  are  negotiations  after  the  establishment 
of  the  competitive  range.  Discussions  are  required  to  be  meaningful  by  indicating 
significant  weaknesses,  deficiencies,  and  other  aspects  that  could  be  altered  so  that  an 
offeror  may  materially  enhance  the  potential  for  award.  Unlike  clarifications,  discussions 
may  result  in  an  offeror  modifying  their  proposal.  [Ref.  1 :  1 5.306] 
C.         IMPROPER  TECHNICAL  EVALUATION 

1.  Cygnus  Corporation 

Cygnus  Corporation,  B-275181,  Comptroller  General  of  the  United  States,  29 
January  1997,  (Cygnus)  is  a  protest  where  GAO  held  a  Federal  agency  evaluation  of 
technical  merit  was  improper;  consequently,  the  Comptroller  General  issued  a 
sustainment.  Cygnus  Corporation  protested  the  Department  of  Health  and  Human 
Services  (HHS)  award  of  a  cost-plus-fixed-fee  contract  for  the  operation  and  management 
of  the  National  Adoption  Information  Clearinghouse.  The  RFP  reserved  the  award  for 
small  businesses  and  outlined  one  base  year  with  two,  one-year  options.  HHS  determined 
the  proposal  of  Caliber  Associates,  Inc.  (Caliber)  to  represent  the  best  value  to  the 
Government. 

The  RFP  stated  that  the  combination  of  the  quality  of  the  technical  proposal  and 
past  performance  was  of  more  importance  than  proposed  cost.  The  solicitation  identified 
five  criteria  for  evaluation:  (1)  quality  of  technical  approach — 30  points;  (2)  staff 
qualifications — 20  points;  (3)  adequacy  of  manpower,  resources,  and  management  plan — 
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20  points;  (4)  understanding  of  the  statement  of  work — 15  points;  and  (5)  corporate 
experience — 15points.  HHR  received  six  proposals  and  found  three  to  be  within  the 
competitive  range.  Caliber  scored  higher  than  Cygnus,  and  based  on  the  results  of  the 
initial  and  BAFO  scores,  the  agency  contract  specialist  prepared  a  Negotiations  Summary 
Memorandum  recommending  award  of  the  contract  to  Caliber.  Caliber  received  the 
award,  and  Cygnus  protested. 

Cygnus  argued  that  HHR  improperly  downgraded  their  proposal  in  the  Quality  of 
Technical  Approach  factor  because  the  agency  failed  to  consider  added  information  from 
the  protester.  The  RFP  required  the  offerors  to  provide  an  indication  of  possible 
problems  they  may  encounter  in  the  performance  of  the  contract  as  well  as  possible 
resolutions.  Cygnus  failed  to  provide  this  information  in  their  proposal.  The  agency 
report  stated  that  the  omission  of  this  information  was  the  main  factor  causing  deduction 
of  points  from  Cygnus'  score.  The  agency  did  not  hold  discussions  because  they  stated 
that  such  discussions  would  result  in  technical  leveling.  However,  Cygnus  did  include 
the  required  information  in  their  BAFO.  The  record  showed  that  Cygnus'  score  was  not 
adjusted  after  the  receipt  of  their  BAFO.  The  evaluator  stated  that  the  information  on  the 
BAFO  was  evaluated  but  was  not  sufficient  to  change  the  score.  GAO  was  skeptical. 
The  score  sheets  still  contained  narrative  stating  that  no  discussions  were  held  about  the 
missing  information  regarding  potential  problems.  The  record  provided  neither 
acknowledgment  of  receipt  of  the  information  nor  evidence  of  its  use  in  evaluating  the 
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BAFO.  For  this  reason,  GAO  held  that  the  HHR  evaluation  of  Cygnus'  proposal  was 
unreasonable  and  that  they  were  prejudiced  because  the  omitted  information  constituted 
the  main  deduction  from  their  score.     GAO  sustained  the  protest  and  recommended 
reevaluation  of  Cygnus'  BAFO  and  comparison  to  the  other  offerors'  BAFOs.  [Ref.  19] 

2.  Matter  of  HG  Properties  A,  L.P. 

Like  Cygnus,  HG  Properties  A,  L.  P.;  B-277572,  B-277572.2,  B-277572.3; 
Comptroller  General  of  the  United  States,  29  October  1997,  (HG)  is  a  protest  that  was 
sustained  because  of  improper  technical  evaluation  by  a  Federal  agency.  HG  Properties 
protested  the  U.S.  Forest  Service  award  of  a  contract  for  over  20,000  square  feet  of  office 
space  for  a  Forest  Service  supervisor's  office  located  2.5  miles  from  Libby,  Montana. 
The  Solicitation  for  Offerors  (SFO)  anticipated  a  10  year  lease  with  two,  five-year 
options.  The  Forest  Service  found  that  Mountain  States  Leasing-Libby  (MSL)  offered  a 
proposal  representing  the  best  value  to  the  Government. 

The  SFO  contained  specifications  outlining  the  requirements  of  the  building 
(architectural,  mechanical,  electrical,  plumbing,  utilities,  maintenance,  and  service 
requirements).  One  written  requirement  was  that  the  computer  room  space  must  be 
located  away  from  areas  housing  microwave  equipment  and  radio  transmitters.  A 
drawing  was  provided—not  to  dictate  design—but  to  show  desired  space  relationships  and 
predicted  traffic  patterns.  The  SFO  also  stated  that  if  any  inconsistency  between  the 
written  requirements  and  the  conceptual  drawing  existed,  the  written  requirements  would 
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govern.  Finally,  the  award  would  be  made  on  a  best  value  basis,  and  technical  evaluation 
factors  were  equal  to  price.  Price  evaluation  was  to  be  on  the  basis  of  total  annual  price 
per  square  foot  of  occupiable  space. 

HG  offered  a  lower  price  per  square  foot  of  $  12.90.  MSL  offered  $  12.93. 
However,  MSL's  overall  point  total  was  better  than  HG's  (846  (MSL)  compared  to  757 
(HB) — 1000  point  maximum).  The  higher  point  total  for  MSL  was  primarily  due  to  their 
high  score  for  efficient  layout.  HG  was  downgraded  because  under  their  layout,  supplies 
could  not  easily  be  moved.  The  agency  determined  that  MSL's  offer  met  all  aspects  of 
the  SFO  while  HG's  did  not  meet  the  basic  floor  plan.  In  fact,  MSL's  layout  was  very 
close  to  the  sample  layout  in  the  SFO.  The  agency  found  that  the  cost  difference  between 
MSL  and  HG  was  minimal,  so  MSL  was  awarded  the  contract  even  though  they  proposed 
a  higher  price  per  square  foot.  HG  protested. 

HG  argued  that  the  Forrest  Service  failed  to  evaluate  MSL's  proposal  as  required 
in  the  SFO.  Specifically,  MSL's  layout  did  not  meet  the  requirement  of  separating  the 
computer  room  from  the  telecommunications  room.  On  the  contrary,  the  computer  room 
was  actually  contained  within  the  telecommunications  room.  The  agency  stated  that 
MSL  had  simply  followed  the  layout  in  the  SFO  and  that  the  minimal  needs  of  the  agency 
were  represented  in  the  drawing.  However,  the  written  direction  in  the  SFO  was  for 
separation  of  the  computer  room  from  telecommunication  and  microwave  equipment. 
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According  to  the  SFO,  the  written  direction  was  overriding  in  the  event  of  a  discrepancy 
with  the  conceptual  drawing. 

GAO  held  that  MSL's  proposed  layout  was  not  rational  and  in  accordance  with 
the  stated  evaluation  criteria  (written  requirements).  The  Comptroller  General  found  that 
HG  had  been  prejudiced  because  they  had  the  lower  price  per  square  foot  and  a  layout 
that  matched  the  SFO  written  direction.  GAO  recommended  that  the  Forest  Service 
amend  their  solicitation  to  reflect  their  actual  minimum  needs  regarding  the  location  of 
the  computer  room,  reopen  negotiations  with  competitive  range  offerors,  and  allow  them 
to  respond  to  the  amended  SFO.  [Ref.  20] 

3.         International  Data  Systems,  Inc. 

International  Data  Systems,  Inc.,  B-277385,  Comptroller  General  of  the  United 
States,  8  October  1997,  (IDS)  is  a  protest  brought  because  the  agency  improperly 
evaluated  the  technical  merit  of  a  delivery  schedule  and  also  failed  to  disclose  proposal 
problems  during  discussions.  International  Data  Systems  protested  the  Department  of 
Interior  (DOI)  award  of  a  fixed-price,  indefinite  delivery /indefinite  quantity  contract  for 
personal  computers.  The  contract  time  period  was  to  span  six  months.  DOI  found 
Applied  Computer  Technology  (ACT)  to  be  the  offeror  representing  the  best  value  to  the 
Government. 

The  RFP  stated  that  a  Technical  Evaluation  Panel  (TEP)  would  consider  the 
following  evaluation  criteria:  conformance  with  technical  specifications — 40  percent, 
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past  performance — 40  percent,  and  price — 20  percent.  Discussions  were  not  to  be  held. 
Forty-two  offers  were  received  with  only  seven  in  the  competitive  range.  The  TEP 
ranked  all  seven  finalists  with  the  protester  earning  first-place  and  ACT  second.  The 
contracting  officer  reviewed  the  seven  proposals  in  greater  detail  and  discovered  that  the 
protester  offered  a  30-45  day  delivery  schedule.  The  RFP  called  for  a  15-day  delivery. 
When  establishing  the  competitive  range,  the  contracting  officer  had  previously  rejected 
other  proposals  that  did  not  meet  delivery  requirements.  She  left  the  protester  in  the 
competitive  range  but  did  not  inform  them  of  the  delivery  problem.  Up  to  this  time,  the 
contracting  officer  believed  that  she  had  held  only  clarifications  with  the  offerors  in  the 
competitive  range.  She  decided  not  to  inform  the  protester  about  the  schedule  problem  in 
order  to  avoid  holding  discussions  and  delaying  the  procurement.  The  DOI  found  ACT's 
proposal  to  be  the  best  value  and  awarded  them  the  contract.  International  Data  Systems 
protested. 

IDS  argued  that  the  agency's  unwillingness  to  inform  them  of  the  problem 
constituted  a  failure  to  hold  meaningful  discussions.  IDS  maintained  that  if  the  problem 
had  been  brought  to  their  attention,  it  could  have  been  immediately  remedied  because 
they  stated  it  was  simply  a  typographical  error.  DOI  countered  that  they  had  engaged  in 
clarifications  with  the  seven  offerors  and  not  discussions.  Consequently,  IDS  was  not 
entitled  to  discussions.  The  governing  regulation  (FAR  Part  1 5 — pre-re write)  permitted 
contracting  agencies  to  award  on  initial  proposals  without  discussions  provided  that  the 
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solicitation  made  the  situation  clear.  Clarifications  were  defined  as  communications  with 
an  offeror  for  the  sole  purpose  of  eliminating  irregularities  or  apparent  clerical  mistakes 
in  a  proposal.  Discussions  involved  essential  information  for  determining  the 
acceptability  of  a  proposal  or  provided  the  offeror  an  opportunity  to  revise  the  proposal. 
Unlike  discussions,  clarifications  do  not  give  an  offeror  the  opportunity  to  make  proposal 
revisions. 

GAO  held  that  the  clarifications  held  by  the  agency  with  some  of  the  offerors 
were  actually  discussions  for  two  reasons.  First,  the  communications  concerned 
information  essential  for  determining  the  acceptability  of  the  offerors'  proposals. 
Second,  the  offerors  were  given  an  opportunity  to  submit  BAFOs  thereby  modifying  their 
proposals.  The  fact  that  discussions  were  held  meant  that  DOI  was  obligated  to  raise  the 
schedule  issue  with  IDS.  When  discussions  are  held,  they  are  required  to  be  meaningful. 
Discussions  are  not  meaningful  unless  they  address  aspects  of  an  offeror's  proposal  that 
must  be  corrected  for  the  offeror  to  have  a  reasonable  chance  of  obtaining  award.  GAO 
held  that  meaningful  discussions  were  required  but  not  held  with  IDS  because  the 
erroneous  delivery  schedule  had  not  been  addressed.  GAO  stated  that  the  lack  of 
required,  meaningful  discussions  coupled  with  an  improper  pre-award  change  by  the 
agency  (discussed  in  a  later  chapter)  prejudiced  IDS.  The  protest  was  sustained.  Since 
the  computers  were  already  delivered  and  accepted,  GAO  recommended  reimbursement 
of  proposal  and  protest  costs  to  IDS.  [Ref.  21] 
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4.  Technology  Services  International,  Inc. 

Technology  Services  International,  Inc.,  B-276506,  Comptroller  General  of  the 
United  States,  21  May  1997,  (TSI)  is  a  protest  in  which  a  proposed  scheduling  system  is 
successfully  challenged.  Technology  Services  protested  the  U.S.  Air  Force  award  of  a 
fixed-price  contract  for  grounds  maintenance  services  at  Hickam  Air  Force  Base,  Hawaii. 
The  contract  was  for  a  six-month  base  period  with  five  option  years.  The  Air  Force 
determined  Oahu  Tree  Experts  (OTE)  to  be  the  offeror  representing  the  greatest  value  to 
the  Government. 

The  RFP  defined  this  as  a  performance  based  contract  with  a  best  value  evaluation 
scheme  consisting  of  the  following  criteria: 

1 .  Technical  (Grounds  Maintenance) 

•  Quality  Control 

•  Equipment,  Material,  and  Supplies  &  Work  Procedures 

•  Customer  Service 

2.  Management  (Manpower  Planning) 

3.  Past  Performance 

4.  Price 

The  first  three  criteria  were  equally  weighted,  and  in  combination,  were  outlined  to  be 
more  important  than  price. 
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OTE  received  an  Unacceptable  rating  in  Technical  (Grounds  Maintenance) — 
Quality  Control  because  the  offeror  did  not  include  a  work  scheduling  system  as  required 
by  the  RFP.  OTE  informed  the  agency  that  such  a  schedule  would  be  developed.  As  a 
result  of  this  promise,  OTE  received  an  Exceptional  rating  (highest  possible)  in  Quality 
Control.  OTE  and  TSI  received  similar  ratings  in  the  various  evaluation  criteria.  OTE's 
proposed  price  of  $4.8  million  was  nearly  $2  million  less  than  the  protester's  $6.5 
million.  Both  proposals  were  found  to  be  essentially  equal  from  a  technical  perspective, 
so  OTE's  lower  price  won  the  contract  award.  Technology  Services  protested. 

The  protester  argued  that  the  agency  technical  evaluation  of  OTE's  proposal  was 
unreasonable.  The  protester  maintained  that  OTE's  BAFO  did  not  provide  a  work 
scheduling  system  that  met  the  requirements  in  the  RFP.  Their  system  failed  to  show  the 
day  and  time  that  the  work  was  to  be  performed.  The  Air  Force  countered  that  OTE  had 
promised  to  develop  such  a  schedule  and  that  their  promise  was  satisfactory.  GAO 
disagreed.    The  solicitation  stated  that  an  Exceptional  rating  was  supposed  to  indicate 
that  a  proposal  had  exceeded  the  standards  outlined  in  the  RFP.  GAO  held  that  the 
assignment  of  such  a  high  rating  was  unreasonable.  Moreover,  the  Comptroller  General 
stated  that  they  could  not  conclude  that  the  two  proposals  were  technically  equal.    GAO 
sustained  the  protest  and  recommended  that  OTE's  contract  be  terminated  and  awarded  to 
TSI.  [Ref.  22] 
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D.         IMPROPER  EVALUATION  OF  TECHNICAL  MERIT  COMBINED 
WITH  INADEQUATE  SUPPORTING  DOCUMENTATION 
1.  Mechanical  Contractors,  S.  A. 

Mechanical  Contractors,  S.  A.,  B-277916,  Comptroller  General  of  the  United 
States,  27  October  1997,  (MECSA)  is  discussed  in  Chapter  IV  because  one  sustaining 
argument  involved  improper  evaluation  of  past  performance.  MECSA  was  also  sustained 
because  of  an  error  by  the  Federal  agency  in  evaluating  technical  merit.    Mechanical 
Contractors  protested  the  Panama  Canal  Commission  (PCC)  award  of  a  firm  fixed-price 
contract  for  cleaning  and  painting  of  four  miter  gate  leaves  in  the  Panama  Canal.  The 
RFP  required  all  offerors  to  submit  a  technical  proposal  and  price  schedule.  The 
technical  proposals  were  to  be  evaluated  under  two  equally  weighted  factors:  technical 

approach  and  performance  capability.  The  total  technical  evaluation  scores  were  to  be 

f —  x-  — 

weighted  equally  with  the  offerors'  firm  fixed-price  in  an  effort  to  determine  the  best 

value  to  the  Government.  MECSA  proposed  the  lowest  price  of  $2.5  million  while  FMS 

submitted  a  price  of  $2.6  million.  Under  technical  approach,  FMS  earned  a  score  of  72.7 

points  compared  to  60.85  for  MECSA.  Despite  MECSA's  lower  price,  FMS  was 

awarded  the  contract. 

The  protester  challenged  the  award  arguing  that  the  agency  technical  evaluation  of 

its  proposal  was  not  adequately  supported.  There  was  very  little  contemporaneous 

supporting  documentation  regarding  MECSA's  technical  evaluation.  Under  technical 
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approach,  the  evaluation  board  stated  only  that  MECSA  had  "very  little  coal  tar  enamel 
(CTE)  experience  over  the  past  three  years."    [Ref.  11] 

Regarding  the  experience  statement,  the  protester  maintained  that  PCC  failed  to 
evaluate  their  proposal  properly.  The  RFP  stated  that  substantially  equal  importance 
would  be  given  to  an  offeror's  CTE  experience  or  an  offeror's  possession  of  a  special 
certification  (SSPCat  QP-2).  The  protester's  proposal  outlined  the  use  of  a  subcontractor 
holding  this  certification.  The  protester  contended  that  since  they  (through  their 
subcontractor)  held  the  QP-2  certification,  which  was  supposed  to  be  substantially  equal 
to  CTE  experience,  their  score  for  technical  approach  should  have  been  essentially  equal 
to  FMS's. 

PCC  countered  that  the  solicitation  did  not  guarantee  equal  or  equivalent 
treatment  of  CTE  experience  and  QP-2  qualification.  PCC  also  asserted  that  the 
evaluation  board  did  give  favorable  consideration  to  the  subcontractor  certification.  GAO 
agreed  that  the  evaluation  board  would  have  been  reasonable  for  a  slight  downgrading  of 
MECSA  regarding  their  limited  direct  CTE  experience.  However,  GAO  held  that  the 
lack  of  contemporaneous  supporting  documentation  cast  doubt  as  to  whether  the 
evaluation  board  actually  gave  MECSA  credit  for  the  subcontractor  certification.  The 
only  documentation  that  accompanied  the  numerical  score  stated  "very  little  CTE 
experience."  GAO  held  that  there  was  no  evidence  of  consideration  of  the  subcontractor 
certification.  GAO  held  that  the  presence  of  the  multiple  material  errors  in  the  evaluation 
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of  technical  approach  and  past  performance  (Chapter  IV)  did  prejudice  MECSA. 
Consequently,  GAO  sustained  the  protest  and  recommended  reevaluation.  [Ref.  1 1  ] 

2.  J.  A.  Jones  Management  Services,  Inc. 

Like  MECSA,  J.  A.  Jones  Services,  Inc.,  B-276874,  Comptroller  General  of  the 
United  States,  24  July  1997,  {Jones)  is  a  protest  where  the  combination  of  improper 
technical  evaluation  and  the  lack  of  proper  agency  documentation  resulted  in  sustainment. 
Jones  Management  Services  protested  the  U.S.  Army  Corps  of  Engineers'  award  of  a 
fixed-price  contract  for  base  repair  and  construction  at  the  Bluegrass  Army  Depot  in 
Lexington,  Kentucky.  The  RFP  called  for  offerors  to  submit  their  best  price/cost  and 
technical  terms  in  the  initial  proposals  because  the  agency  intended  to  issue  award 
without  discussions.  The  contract  was  to  be  for  an  1 8-month  period  with  two  option 
years.  The  U.S.  Army  Corps  of  Engineers  (USACOE)  found  Intersteel,  Inc.  (Intersteel) 
to  be  the  offeror  whose  proposal  represented  the  best  value  Government. 

The  solicitation  listed  the  following  evaluation  criteria  (descending  order  of 
importance):  (1)  management  ability;  (2)  subcontracting  support  capability;  (3)  related 
experience;  (4)  unit  price  coefficient;  (5)  technical  staff  capability;  and  (6)  financial 
ability.  Regarding  price,  each  offeror  was  required  to  include  a  coefficient  or  percentage 
factor  to  be  applied  to  specified  fixed  rates  for  various  prices  contained  in  a  unit  price 
book  (enclosed  with  the  RFP).  Jones  Management  Services  challenged  the  award  stating 
that  the  majority  of  the  technical  point  ratings  were  inaccurate,  unsupported,  or  otherwise 
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improperly  based  on  undisclosed  evaluation  criteria.  Jones  Management  Services 
maintained  that  the  agency  evaluation  and  supplemental  report  provided  explanations 
with  information  that  was  either  cursory,  missing,  or  "squarely  at  odds"  with  the  proposal 
information.  The  record  revealed  that  the  only  supporting  documentation  consisted  of 
bullet  statements  annotated  on  the  score  sheets.  In  addition,  the  scant  cursory  narratives 
did  not  match  the  scoring  deductions.  The  protester  charged  that  strengths  and 
weaknesses  assigned  to  the  offerors  bore  no  relationship  to  the  evaluation  criteria  outlined 
in  the  RFP.  To  further  support  its  position,  the  protester  pointed  that  the  evaluation 
record  did  not  reference  a  single  paragraph  of  either  the  protester  or  the  awardee 
proposals. 

Regarding  the  allegation  of  inaccurate  evaluation,  the  agency  countered  that  the 
bullet  narratives  on  the  score  sheets  were  only  impressions  and  did  not  wholly  represent 
the  justification  for  the  scores.  However,  GAO  held  that  exclusion  of  these  bullets  left 
the  agency  with  absolutely  no  supporting  documentation.  The  Comptroller  General  held 
that  the  Corps  of  Engineers'  evaluation  was  both  inaccurate  and  unsupported  by  adequate 
documentation,  narrative,  or  information.     GAO  continued  stating:  "Here,  the  dearth  of 
evaluation  narratives  or  other  point  scores,  which  have  been  sufficiently  shown  by  the 
protester  to  be  inaccurate  in  numerous  areas,  lead  us  to  the  conclusion  that  that  the  Source 
Selection  Authority  could  not  and  did  not  make  a  reasonable  selection  decision  . .  .the 
agency  has  offered  no  substantive  response  to  the  protester's  concerns  in  response  to  the 
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protest."  GAO  sustained  the  protest  and  recommended  reevaluation  of  all  proposals. 
[Ref.  1 7] 

3.  JW  Associates,  Inc. 

JW  Associates,  Inc.,  B-275209,  Comptroller  General  of  the  United  States,  30 
January  1997,  (JW)  is  the  third  protest  in  this  chapter  where  poor  evaluation 
documentation  was  the  basis  for  a  sustaining  argument.  JW  Associates  protested  the 
Forest  Service  award  of  a  small  business  set-aside  contract  for  an  environmental  impact 
statement  for  the  Cold  Springs  Analysis  Area  on  the  Medicine  Bow  National  Forest  in 
Wyoming.  The  Forest  Service  determined  that  Natural  Resources  Management 
Corporation  (NRMC)  offered  the  proposal  representing  the  best  value  to  the  Government. 

The  RFP  listed  five  evaluation  criteria  in  the  order  of  importance  (descending 
order  of  importance)  with  the  first  two  categories  termed  as  Very  Important  and  the  last 
three  as  Important: 

Very  Important 

1 .  Qualifications  of  the  Firm 

2.  Qualification  of  the  Personnel 
Important 

3.  Past  Experience  of  the  Firm  and  the  Employees  Assigned  to  the  Project 

4.  Geographic  Location 

5.  Price 
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The  solicitation  also  stated  that  the  award  would  be  based  on  a  technical/cost  tradeoff 
within  the  pool  of  technically  acceptable  proposals.  Each  member  of  the  evaluation  board 
(made  up  of  three  forest  service  experts)  assigned  technical  scores  to  the  criteria  for  each 
proposal.  The  three  scores  were  averaged  for  each  offer,  and  the  proposals  were  ranked. 
JW  was  the  highest  technically  rated  offeror  but  ranked  sixth  in  price.  NRMC  ranked 
first  in  price.  The  board  prepared  a  memorandum  for  the  record  containing  a  one- 
paragraph  summary  of  each  proposal.  The  contracting  officer  concluded  that  NRMC's 
proposal  represented  the  best  value  to  the  Government  and  made  the  award.  JW  received 
a  debriefing  letter  along  with  a  copy  of  the  awardee's  technical  proposal.  JW  protested. 

JW  argued  that  the  Forest  Service's  evaluation  of  the  offerors'  technical  proposals 
was  improper  and  inconsistent  with  the  evaluation  criteria  outlined  in  the  solicitation. 
Under  the  most  heavily  weighted  criterion  Qualifications  of  the  Firm,  JW  asserted  that  it 
had  two  recent  timber  sale  environmental  impact  statements  and  was  working  on  a  third 
at  the  time  of  evaluation.  JW  also  maintained  that  NRMC  had  no  experience  regarding 
environmental  impact  statements.  JW  concluded  that  its  score  for  firm  qualifications 
should  have  been  greater  that  NRMC's  score. 

The  agency  stated  that  the  evaluation  was  fairly  performed  and  in  keeping  with 
the  provisions  in  the  RFP.  In  response  to  specific  assertions  by  the  protester,  the  Forest 
Service  only  responded  by  repeating  the  evaluation  criteria  and  the  method  for  averaging 
scores.  GAO  found  that  the  technical  evaluation  was  not  adequately  supported.  The 
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Comptroller  General  continued  stating  that  without  adequate  supporting  documentation 
for  the  evaluation,  a  proper  award  determination  was  not  possible.    Furthermore,  the 
contracting  officer  debriefing  letter  stated  plainly  that  price  was  the  basis  for  the  agency's 
award  decision.  The  RFP  stated  that  price  was  last  in  importance  and  only  the  deciding 
factor  if  two  proposals  were  found  to  be  technically  equal.  GAO  held  that  the  record  did 
not  contain  sufficient  contemporaneous  documentation,  information,  or  analysis  to  make 
a  decision  of  technical  equality.  GAO  stated  that  the  FAR  15.612(d)(2)  (pre-rewrite) 
required  that  the  documentation  supporting  selection  decisions  show  the  relative 
differences  among  proposals;  their  strengths,  weaknesses,  and  risks;  and  the  basis  for 
award  decisions.  GAO  held  that  the  Forest  Service  did  not  adhere  to  the  regulation  and 
sustained  the  protest.  The  Comptroller  General  recommended  discussions  (if  deemed 
necessary  by  the  agency),  revised  BAFOs,  and  reevaluation.  [Ref.  1 8] 
E.         ANALYSIS  AND  CONCLUSION 

The  purpose  of  this  analysis  is  to  give  the  contracting  officer  an  indication  of  the 
common  pitfalls  when  evaluating  technical  merit  in  best  value  contracts. 

1.  Craft  An  Accurate  Solicitation  That  Reflects  the  True  Agency 

Requirement 

A  best  value  source  selection  can  be  undermined  early  in  the  contracting  process 
by  way  of  a  sloppily  crafted  solicitation.  HG  is  a  prime  example  where  the  true  desires  of 
an  agency  are  not  reflected  in  the  RFP.  The  Forest  Service  stated  during  the  protest  that  a 
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conceptual  drawing  of  the  supervisor's  office  met  the  minimum  needs  of  the  agency. 
Contrary  to  the  drawing,  the  solicitation  stated  in  writing  that  the  computer  room  must  be 
away  from  microwave  and  radio  transmitting  equipment.  These  contradictory 
requirements  resulted  in  the  agency  essentially  ignoring  their  written  (overriding) 
requirement  when  making  their  award  decision.  The  Comptroller  General  analysis  and 
decision  were  obvious  in  this  case.  GAO  held  that  the  agency  and  awardee  did  not  follow 
the  solicitation.  This  holding  stems  from  a  previous  protest  decision  (Southwest  Marine, 
Inc.,  American  Systems  Engineering  Corporation;  B-265865.3,  B-265865.4;  23  January 
1996)  where  GAO  asserted  that  the  evaluation  of  proposals  must  be  reasonable  and  bear  a 
rational  relationship  to  the  announced  criteria  upon  which  competing  offers  are  to  be 
selected.  [Ref.  20]    The  implication  for  the  contracting  officer  is  that  great  care  must  be 
taken  to  ensure  the  solicitation  is  well  written  and  reflects  the  actual  customer 
requirement.  Poorly  crafted  solicitations  are  susceptible  to  protest  and  sustainment. 
2.  Evaluate  Proposals  in  Accordance  with  the  Solicitation 

The  inability  or  unwillingness  to  follow  the  evaluation  criteria  in  the  solicitation  is 
a  prevalent  miscue  by  Federal  agencies.  TSI  documents  an  improper  technical  evaluation 
because  a  Federal  agency  chose  not  to  follow  the  solicitation.  Technology  Services 
protested  the  award  of  a  Hawaiian  grounds  maintenance  contract  in  which  the  awardee 
was  not  held  to  the  requirements  of  the  RFP  by  the  agency.  The  RFP  required  a  work 
scheduling  system  for  which  the  awardee  issued  only  a  promise  to  develop.  The  agency 
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(Air  Force)  accepted  the  promise  and  changed  the  awardee's  score  from  the  lowest 
possible  {Unacceptable)  to  the  highest  attainable  {Exceptional)  rating.  However,  the 
good  will  and  trust  of  the  Air  Force  were  not  enough  to  convince  the  Comptroller 
General.  The  protester  successfully  argued  that  the  awardee  simply  had  not  met  the 
requirements  of  the  RFP.  GAO  agreed.  Rather  than  the  typical  recommendation  of 
reevaluation,  GAO  held  that  the  awardee's  proposal  was  unacceptable  and  recommended 
termination  and  immediate  award  to  the  protester.  This  is  despite  the  fact  that  the 
protester's  proposed  price  was  nearly  $2  million  greater  that  the  proposed  price  of  the 
awardee.  This  holding  stemmed  from  a  previous  protest  decision  {Tidewater  Homes 
Reality,  Inc.,  B-274689,  26  December  1996)  where  the  Comptroller  General  held  that  an 
agency  evaluation  may  not  lack  a  reasonable  basis  or  conflict  with  the  stated  evaluation 
criteria  for  award.  [Ref.  22]  The  implication  for  contracting  officers  is  that  they  must 
adhere  to  the  evaluation  criteria  set  forth  in  the  solicitation  when  measuring  technical 
merit.  They  must  also  thoroughly  research  solicitations  to  ensure  they  are  free  from 
contradictions  and  ambiguities.  The  failure  to  do  so  can  result  in  unfair  and  unreasonable 
contract  award  or  perhaps  the  purchase  of  an  undesirable  product  or  service  by  an  agency. 
3.        Apply  Supporting  Narratives  to  Evaluation  Records 
The  requirement  to  attach  narratives  to  the  previously  required  score  sheets  and 
rankings  is  a  product  of  the  FAR  Part  15  rewrite.  However,  GAO  stressed  the  need  for 
supporting  documentation  in  three  pre-rewrite  protests. 
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MECSA  is  a  protest  that  challenged  a  Federal  agency  award  of  a  contract  for  the 
cleaning  and  painting  of  four  miter  gates  in  the  Panama  Canal.  The  agency's  technical 
evaluation  was  undermined  by  the  lack  of  contemporaneous  supporting  documentation. 
The  agency  had  no  justification  in  the  evaluation  record  for  overriding  the  protester's 
lower  price  and  awarding  to  Formal  Management  Systems  (FMS).  In  addition,  the  lack 
of  supporting  statements  called  into  question  whether  the  agency  actually  accounted  for  a 
special  certification  (QP-2)  held  by  one  of  the  protester's  subcontractors.  The  agency 
maintained  that  it  had  given  the  subcontractor  certification  proper  consideration.  They 
may  have,  but  the  record  did  not  support  their  assertion.  As  a  result,  GAO  found  that  the 
protester  had  been  prejudiced  and  sustained  the  protest. 

Jones  is  a  protest  that  reaffirms  the  crippling  need  for  narrative  in  best  value 
source  selection.  J.  A.  Jones  protested  the  Army  Corps  of  Engineers  award  of  a  contract 
for  base  repair  and  construction.  The  agency  had  only  score  sheets  with  bullet  statements 
to  support  their  evaluation.  When  their  evaluation  was  found  to  be  unreasonable  and 
inaccurate,  they  claimed  that  the  bullets  were  only  impressions  and  not  the  full 
explanation  as  to  why  they  selected  Intersteel  Inc.  as  the  awardee.  GAO  held  that  if  the 
bullets  did  not  represent  their  rationale  for  selecting  Intersteel,  then  their  level  of 
documentation  decreased  from  little  to  none.  The  Army  had  committed  the  double  error 
of  improperly  evaluating  the  proposals,  and  providing  little  or  no  documentation  to 
justify  their  selection.  GAO  termed  the  amount  of  supporting  documentation  as  a  "dearth 
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of  evaluation  narratives."  The  Corps  of  Engineers  left  GAO  with  little  choice  but  to 
sustain  the  protest  and  recommend  reevaluation. 

JW  is  a  third  protest  where  lack  of  supporting  documentation  resulted  in 
sustainment.  The  agency  in  this  case  clearly  did  not  meet  the  FAR  Part  1 5  pre-rewrite 
standard  requiring  strengths,  weaknesses,  risks,  and  basis  for  award  to  be  included  in  the 
evaluation  record.  The  agency  (Forest  Service)  appeared  to  have  improperly  evaluated 
both  the  protester  and  awardee  in  the  Quality  of  Firms  factor.  The  protester  had  two 
complete  and  one  in  progress  environmental  impact  statements  while  the  awardee  had 
none.  Still,  the  awardee  received  a  higher  score  in  this  heavily  weighted  factor.  To  make 
matters  worse,  the  only  supporting  documentation  consisted  of  a  one  paragraph  per 
offeror  narrative  that  in  no  way  explained  the  scoring  of  the  factor  in  question  or  the 
specific  reasons  for  award  selection.  GAO  sustained  the  protest. 

MECSA,  Jones,  and  JW  demonstrate  that  the  nature  of  tradeoff  is  not  always  clear 
and  sometimes  tenuous.  The  use  of  narrative  comments  provides  the  agency  with  an 
opportunity  to  explain  fully  their  reasons  for  scoring,  ranking,  and  source  selection.  The 
requirement  for  narrative  comments  (in  addition  to  strengths,  weaknesses,  risks,  and  basis 
for  award)  was  not  in  force  when  these  contracts  were  awarded,  yet  GAO  appeared 
compelled  to  send  the  message  that  narratives  are  important.  The  implication  for 
contracting  officers  is  that  they  must  now  make  liberal  use  of  narrative  because  the  FAR 
Part  1 5  rewrite  mandates  it.  More  importantly,  it  allows  a  Federal  agency  to  announce  its 
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methods,  motives,  and  reasons  for  tradeoff  and  source  selection.  The  key  fact  is  that 
contracting  officers  will  be  well  served  if  they  supply  narratives  with  best  value  awards 
that  include  the  agency's  very  thought  processes  in  arriving  at  a  selection  decision. 
Failure  to  convey  the  rationale  for  tradeoff  and  source  selection  adequately  could  be 
perceived  by  GAO  as  unreasonable  technical  evaluation  and  source  selection. 

4.        Hold  Meaningful  Discussions  Regarding  Technical  Evaluation  When 
Required 

IDS  is  a  protest  that  could  have  been  avoided  through  the  use  of  meaningful 
discussions.  Discussions  both  before  and  after  the  FAR  Part  1 5  rewrite  involve  essential 
information  for  determining  the  acceptability  of  a  proposal  or  provide  the  offeror  an 
opportunity  to  revise  the  proposal.  The  protester  maintained  that  its  typographical  error 
in  delivery  schedule  (30-45  days  instead  of  the  required  15)  could  have  been  rectified 
through  discussions.  The  RFP  stated  discussions  would  not  be  held,  and  the  agency 
believed  all  dialog  up  to  that  point  had  been  clarifications.  GAO  held  that  the 
clarifications  were  actually  discussions  and  sustained  the  protests.  This  decision  was 
based  in  the  FAR  15.601  (pre-rewrite)  statement  that  defines  discussions  as  exchanges 
with  offerors  that  involve  key  information  for  determining  the  acceptability  of  a  proposal. 
In  addition,  all  the  clarifications  qualified  as  discussions  because  BAFOs  were  issued  and 
accepted  as  a  result,  so  the  offerors  were  provided  an  opportunity  to  modify  their 
proposals.  The  presence  of  discussions  then  meant  that  they  had  to  be  meaningful.  GAO 
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held  that  the  discussions  with  the  protester  were  not  meaningful  because  they  did  not 
address  aspects  of  an  offeror's  proposal  that  must  be  corrected  for  the  offeror  to  have  a 
reasonable  chance  to  obtain  award.  The  agency  failed  to  address  the  delivery  schedule; 
therefore,  the  discussions  were  not  meaningful.  The  implication  for  contracting  officers 
is  that  all  exchanges  with  offerors  must  be  scrutinized  and  identified  (e.g.  discussions, 
clarifications).  If  discussions  are  held,  they  must  be  meaningful  and  conducted  with  all 
offerors  in  the  competitive  range.    The  Part  15  rewrite  has  opened  the  door  for  increased 
dialog  through  communications  during  competitive  range  determination  and  for  adverse 
past  performance  references.  This  forces  contracting  officers  to  be  even  more  aware  of 
the  consequences  of  their  exchanges  with  offerors. 

5.  Screen  all  Offeror  Documents  for  Relevant  Technical  Information 

Cygnus  is  a  protest  that  should  never  have  happened.  Cygnus  Corporation 
protested  the  award  of  a  contract  operation  and  management  of  the  National  Adoption 
Information  Clearinghouse  by  the  Department  of  Health  and  Human  Services.  Here,  the 
agency  apparently  overlooked  or  forgot  to  include  information  (potential  contract 
problems  and  solutions)  from  the  protester's  BAFO  in  the  technical  scoring.  The  record 
contained  no  indication  that  the  information  had  been  received,  evaluated,  or  included  on 
the  scoring  sheets.  GAO  found  the  technical  evaluation  unreasonable  and  sustained  the 
protest.  The  lesson  for  contracting  officers  is  simple.  Carefully  review  all  documents 
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from  offerors  for  pertinent  information.  Failure  to  do  so  could  result  in  an  embarrassing 
protest  for  which  a  Federal  agency  has  no  defense. 

Contracting  officers  can  learn  a  great  deal  from  this  chapter.  First,  crafting  an 
accurate  and  clear  solicitation  is  vital.  Second,  folio  wing  the  solicitation  is  critically 
important.  Third,  narratives  in  conjunction  with  other  supporting  documentation  can 
explain  an  agency's  rationale  for  technical  tradeoff  and  best  value  source  selection. 
Fourth,  meaningful  discussions  about  a  technical  evaluation  may  be  necessary  to  allow  an 
offeror  to  correct  a  proposal  and  have  a  reasonable  chance  to  obtain  award.  Finally,  all 
offeror  correspondence  must  be  carefully  reviewed  for  relevant  technical  information  that 
could  affect  the  outcome  of  the  source  selection  decision. 
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VI.       COST/PRICE 
A.         INTRODUCTION 

Cost  or  price  is  one  of  the  most  important  tradeoff  elements  regarding  best  value. 
The  reason  is  simple.  Cost/Price  must  be  evaluated  in  all  proposal  evaluations  as  a  matter 
of  regulation.  Cost/Price  is  especially  important  in  best  value  contracting  because  it  is  the 
single  element  against  which  all  other  elements  (past  performance,  technical  merit,  etc.) 
are  traded  to  obtain  best  value.  The  six  sustained  protests  in  which  improper  evaluation 
of  price  or  cost  was  a  sustaining  argument  are:  The  Arora  Group,  Inc.,  Barents  Group, 
L.L.  C. ;  Boeing  Sikorsky  Aircraft  Support;  Geo-Centers,  Inc. ;  Sylvest  Management 
Systems  Corporation;  and  Tri-State  Government  Services,  Inc.  These  protests  are  divided 
into  three  categories: 

1 .  Agency  failure  to  evaluate  the  total  proposal  cost. 

2.  Improper  agency  evaluation  of  cost  realism  and/or  price  realism. 

3.  Agency  failure  to  follow  the  solicitation. 

Three  of  the  protests  were  also  sustained  for  improper  agency  discussions  regarding  the 
evaluation  of  price  or  cost.  This  chapter  discusses  the  affect  of  the  FAR  Part  15  rewrite 
on  cost  or  price  evaluation,  describes  the  six  sustained  protests,  and  provides  analysis 
about  Federal  agency  miscues  when  evaluating  cost  or  price. 
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B.         COST/PRICE  AND  FAR  PART  15 

The  Part  1 5  rewrite  does  not  make  any  significant  changes  regarding  price  or  cost. 
As  previously  stated,  cost/price  is  required  to  be  evaluated  in  every  source  selection. 
Cost/price  is  often  analyzed  to  ensure  a  fair  and  reasonable  price  for  the  Government. 
Competition  normally  establishes  price  reasonableness.  With  fixed-price  contracts, 
comparison  of  the  proposed  prices  usually  satisfies  the  requirement  for  price  analyses. 
Cost-reimbursement  proposal  evaluations  must  include  cost  realism  analyses  to  determine 
what  the  Government  should  expect  to  pay  for  proposed  goods  or  services.  [Ref.  1 : 
15.304,  15.305]  The  key  behind  the  rewrite  is  the  increased  emphasis  on  best  value 
contracting.  The  result  is  a  different  and  sometimes  more  difficult  handling  of  price  or 
cost  in  proposal  evaluation.  Before  the  widespread  use  of  best  value  contracting,  most 
contracts  were  awarded  by  virtue  of  being  either  lowest  price-technically  acceptable  or 
highest  technically  rated.  Best  value  pushes  the  contracting  officer  within  this  spectrum 
and  cost  or  price  must  always  be  a  tradeoff  element.  The  contracting  officer  must 
exercise  good  judgment  in  trading  off  non-cost  factors  (technical  merit,  past  performance, 
labor  qualifications,  etc.)  against  price  or  cost.  Increased  best  value  awards  mean  more 
complexity  in  the  contracting  officer's  proposal  evaluations.  Cost/price  is  a  key  issue  for 
the  contracting  officer  to  manage  on  every  proposal  evaluation. 
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C.         AGENCY  FAILURE  TO  EVALUATE  THE  TOTAL  PROPOSED  COST 
OR  PRICE 
1.  Boeing  Sikorsky  Aircraft  Support 

Boeing  Sikorsky  Aircraft  Support;  B-277263.2,  B-27263.3;  Comptroller  General 
of  the  United  States;  29  September  1997,  {Boeing)  is  a  protest  in  which  agency  failure  to 
evaluate  the  total  proposed  cost,  improper  agency  evaluation  of  labor  cost,  and  agency 
failure  to  hold  meaningful  discussions  resulted  in  sustainment.  Boeing  protested  the  U.S. 
Special  Operations  Command  (SOCOM)  award  of  a  cost-plus-award-fee  contract  for  the 
operations  and  maintenance  of  the  Special  Operations  Forces  Support  Activity  (SOFSA) 
at  depots  in  Richmond  and  Lexington,  Kentucky.  SOFSA  consists  of  Government- 
owned,  contractor-operated  facilities  that  provide  logistics  support  for  the  Special 
Operations  Forces  (SOF)  including  equipment  repair,  modification,  and  sustainment; 
prototype  and  low  volume  manufacturing;  maintenance  management;  and  life  cycle 
support.  The  contract  was  for  one  base  year  with  four,  one-year  options.  SOCOM 
determined  Raytheon  E-Systems  (Raytheon)  to  be  the  offeror  whose  proposal  represented 
the  best  value  to  the  Government. 

The  RFP  was  written  to  allow  offerors  as  much  latitude  as  possible  to  propose 
innovative  ways  of  doing  business  and  to  encourage  creativity  in  problem  solving. 
Offerors  were  required  to  complete  seven  sample  tasks.  Six  of  the  tasks  were  outlined  in 
the  RFP,  and  the  seventh  was  to  be  performed  within  two  weeks  of  the  closing  date  to 
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simulate  the  rapid  reaction  environment  of  SOFSA.  The  proposals  were  evaluated  in  the 
following  areas  (descending  order  of  importance):  (1)  technical,  (2)  management,  (3) 
performance  risk,  and  (4)  cost.  The  technical  area  was  comprised  of  the  seven  required 
tasks  where  management  and  performance  risk  each  had  various  sub-factors  for 
evaluation.  Cost  evaluation  encompassed  five  sub-factors:  (1)  cost  estimating  and 
control,  (2)  quality  control  (3)  security  management,  (4)  schedule  planning  and  control, 
and  (5)  management  effectiveness.  The  cost  evaluation  scheme  used  a  risk  scale  (low, 
moderate,  high)  to  score  the  various  sub-factors.  This  cost  evaluation  structure  was 
designed  to  measure  each  offeror's  ability  to  estimate  accurately  the  cost  of  completing 
logistics  support  tasks,  as  well  as  each  offeror's  overall  proposal  cost.  Five  offers  were 
received  by  SOCOM.  SOCOM  found  Raytheon  to  be  equal  to  Boeing  in  technical  merit 
and  performance  risk  and  superior  to  Boeing  in  the  management  and  cost  factors. 
SOCOM  awarded  the  contract  to  Raytheon.  Boeing  Sikorsky  protested. 

The  protester  raised  two  challenges  to  SOCOM' s  cost  evaluation.  First,  the 
protester  argued  that  the  agency  failed  to  take  into  account  the  most  probable  cost  of  the 
proposals.  In  other  words,  SOCOM  evaluated  cost  only  from  a  risk  perspective — not 
total  cost  difference  between  competing  proposals.  The  agency  countered  by  stating  that 
it  was  not  required  to  consider  the  most  probable  costs  because  the  solicitation  made  no 
such  provision.  GAO  showed  that  the  RFP  stated  that  the  total  dollar  figure  would  be 
used  by  the  Government  as  an  indicator  of  total  cost  for  the  expected  term  of  the  contract. 
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Even  if  the  solicitation  supported  the  agency,  GAO  found  SOCOM's  assertion  that 
consideration  of  total  cost  was  not  a  requirement  to  be  legally  incorrect.    GAO  stated  that 
CICA  (1984)  mandated  inclusion  of  cost  or  price  as  a  significant  factor  in  the  evaluation 
of  proposals.  GAO  determined  that  SOCOM  failed  to  give  significant  consideration  to 
cost  and  found  their  evaluation  to  be  unreasonable. 

The  protester's  second  argument  maintained  that  SOCOM  erroneously  increased 
its  labor  costs.  The  protester  had  failed  to  consolidate  all  labor  costs  into  one  figure  in  its 
proposal  in  accordance  with  the  RFP.  Even  though  the  summation  of  labor  cost  was  not 
in  the  proposal,  there  were  no  missing  labor  hours  in  the  offer.  As  a  result,  the  agency 
erroneously  adjusted  Boeing's  cost  upward  to  compensate  for  what  appeared  to  be 
missing  labor  hours.  GAO  found  that  the  accuracy  of  the  labor  hour  assessment  was  not 
the  critical  issue.  The  key  issue  regarding  the  labor  hours  was  that  SOCOM  failed  to 
conduct  meaningful  discussions  with  Boeing  regarding  a  perceived  significant  error  in 
their  proposal.  Boeing  should  have  been  placed  on  notice  regarding  this  perceived 
weakness  in  their  proposal.  The  resultant  cost  adjustment  was  significant,  for  Boeing  had 
been  assigned  a  high  risk  rating  in  overhead  and  labor  cost  evaluations.  Moreover,  the 
elimination  of  the  cost  adjustment  through  discussions  would  have  resulted  in  Boeing 
having  a  lower  cost  proposal  than  Raytheon. 

The  lack  of  discussions  was  the  linchpin  in  this  protest.  The  failure  to  evaluate 
the  total  cost  would  not  normally  have  been  crucial  in  this  particular  award  because 
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Raytheon  had  a  proposal  with  an  overall  advantage  in  non-cost  factors  as  well  as  cost 
factors.  However,  with  the  agency  error  regarding  the  assessment  and  subsequent  upward 
adjustment  to  labor  hours,  Boeing  may  well  have  been  denied  cost  advantage  due  to  the 
evaluation.  A  Boeing  cost  advantage  coupled  with  a  Raytheon  non-cost  advantage  would 
have  required  the  agency  to  conduct  a  tradeoff  analysis  to  see  which  proposal  represented 
the  best  value  to  the  Government.  In  this  case,  no  such  tradeoff  analysis  was  executed. 
GAO  found  that  Boeing  had  been  prejudiced  by  the  combination  of  agency  failure  to 
compare  total  costs,  agency  error  in  evaluating  labor  cost,  and  agency  failure  to  hold 
meaningful  discussions.  The  Comptroller  General  sustained  the  protest.  GAO 
recommended  that  SOCOM  reopen  negotiations  with  all  offerors,  request  revised 
proposals,  and  reevaluate  the  proposals  for  a  best  value  award.  [Ref.  24] 

2.  Sylvest  Management  Systems  Corporation 

Sylvest  Management  Systems  Corporation;  B-275935,  B-275935.2;  Comptroller 
General  of  the  United  States;  21  April  1997,  (Sylvest)  is  a  protest  where  agency  failure  to 
evaluate  the  total  price  (all  costs)  resulted  in  sustainment.  Sylvest  protested  the 
Tennessee  Valley  Authority  (TV  A)  award  of  a  contract  for  the  purchase  of  computer 
hardware  and  maintenance  support  services.  TVA  determined  BTG  Inc.  (BTG)  to  be  the 
firm  whose  proposal  represented  the  greatest  value  to  the  Government. 

The  RFP  described  three  schedules  of  required  services.  Schedule  I  included  the 
purchase  of  computer  maintenance  services.  Schedule  II  and  III  were  for  the  purchase  of 


80 


new  hardware  plus  various  hardware  and  software  upgrades  to  TVA's  current  computer 
system.  The  solicitation  stated  that  multiple  contracts  could  be  issued  to  satisfy  the 
requirements  contained  within  the  three  schedules.  This  protest  only  addressed  the 
TVA's  award  under  Schedule  III  for  new  hardware.  Under  Schedule  III,  each  offeror  was 
required  to  submit  offers  for  nine  servers  of  three  sizes  (small,  medium,  and  large)  from 
three  separate  manufacturers.  The  offerors  were  required  to  test  and  certify  that  the 
hardware  met  both  industry  standards  and  TVA-specific  benchmarks.  The  RFP  stated 
that  the  contract  would  be  awarded  to  the  offeror  who  received  the  highest  cumulative 
score  for  both  technical  and  price  evaluation  factors.  TVA  evaluated  all  proposals,  held 
discussions,  received  two  rounds  of  BAFOs,  and  awarded  a  contract  under  Schedule  III  to 
BTG.  Sylvest  Management  Systems  protested. 

The  protester  argued  that  the  agency  erroneously  evaluated  BTG's  price  proposal. 
The  protester  asserted  that  the  BTG  did  not  propose  the  same  hardware  configuration  that 
they  used  to  test  to  TVA  benchmark  standards  successfully.  The  protester  also  stated  that 
a  significant  amount  of  memory  had  to  be  added  to  BTG's  basic  (proposed)  hardware 
configuration  to  meet  TVA  benchmarks  successfully.  The  protester  pointed  out  that  BTG 
had  listed  the  extra  hard  disk  memory  as  an  option  in  the  proposal,  and  as  a  result,  the 
agency  did  not  consider  all  costs  (cost  of  required  extra  memory)  when  determining  best 
value. 
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TV  A  countered  stating  that  the  certification  in  BTG's  BAFO  was  a  satisfactory 
indication  that  BTG  was  offering  the  optional  additional  memory  as  part  of  its  basic 
package.  GAO  described  TVA's  position  as  untenable.  The  Comptroller  General  found 
(without  yet  evaluating  the  certification)  that  it  was  clear  from  BTG's  BAFO  that  the 
extended  price  did  not  include  the  extra  hard  disk  memory  required  to  satisfy  TVA 
benchmarks.  GAO  further  held  that  the  certification  did  not  incorporate  the  required 
additional  memory  in  the  offered  system  at  no  additional  cost.  BTG's  certification  stated 
only  that  their  proposal  had  been  updated  to  reflect  the  benchmarked  configuration.  The 
awardee's  certification  did  not  state  that  all  items  included  in  the  required  configurations 
were  included  in  the  systems  prices  as  required  by  the  solicitation.  The  Comptroller 
General  also  found  that  BTG's  exclusion  of  the  substantial  cost  required  to  upgrade  the 
hard  disk  resulted  in  BTG  having  a  lower  cost  than  Sylvest.  GAO  found  that  Sylvest  had 
been  prejudiced  and  recommended  reevaluation  of  the  BAFOs  submitted  in  response  to 
Schedule  III.  [Ref.  26] 
D.  IMPROPER  AGENCY  EVALUATION  OF  COST  REALISM  AND/OR 

PRICE  REALISM 

1.  Barents  Group  L.L.C 

Barents  Group,  L.L.C;  B-276082,  B27  6082.2;  Comptroller  General  of  the  United 
States;  9  May  1997,  {Barents)  is  a  protest  where  improper  agency  cost  realism  analysis 
failed  to  detect  awardee  conditions  placed  on  RFP  mandated  pricing  multipliers.  The 
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result  was  sustainment.  Barents  protested  the  award  of  an  Agency  for  International 
Development  (AID)  contract  for  technical  expertise  in:  (1)  economic  and  institutional 
analysis  and  (2)  private  sector  development  issues.  The  solicitation  envisioned  between 
four  and  six  indefinite  quantity  contracts.  Each  of  these  contracts  was  to  be  for  three 
base-years  and  two  option  years  with  one  contract  set  aside  under  the  Small  Business 
Administration's  8(a)  program.  AID  found  Chemonics  International,  Inc.  (Chemonics); 
Booz- Allen  &  Hamilton,  Inc.  (Booz-Allen);  Abt  Associates,  Inc.  (Abt);  Carana 
Corporation  (Carana);  and  Deloitte  Touche  Tohmatsu  (Deloitte)  to  be  the  offerors  who 
submitted  proposals  representing  the  greatest  value  to  the  Government. 

The  RFP  outlined  the  award  of  contracts  in  two  functional  areas;  however, 
protests  were  issued  regarding  only  line  item  0001 — privatization  issues.  The  solicitation 
stated  that  awards  would  be  made  to  the  responsible  offerors  submitting  acceptable, 
reasonably  priced  proposals  that  offered  best  value  to  the  Government.  Each  proposal 
was  to  be  assigned  a  score  for  technical  and  cost.  Technical  was  weighted  60  percent  and 
cost  40  percent.  Technical  factors  included  the  following  (25  points  each):  (1)  personnel 
qualifications  and  experience,  (2)  quality  and  responsiveness,  (3)  demonstrated  corporate 
experience,  and  (4)  past  performance. 

Under  cost,  the  solicitation  called  for  offerors  to  propose  a  maximum  fixed  daily 
salary  for  seven  labor  categories  covering  the  base  and  option  periods.  In  addition,  the 
offerors  were  supposed  to  issue  two  fixed  multipliers,  one  for  U.S.  expatriate  staff  and  the 
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other  for  non-U. S.  personnel.  The  multipliers  were  to  contain  all  payroll  costs,  indirect 
costs,  home/corporate  office  secretarial/administrative  support,  computer  rental,  report 
preparation  costs,  and  profit  or  fee.    The  solicitation  stated  that  the  agency  would 
calculate  an  average  burdened  daily  rate  using  the  multipliers  and  the  maximum  fixed 
daily  salaries. 

AID  received  1 7  offers.  Agency  officials  then  scored  the  offers  under  the  four 
technical  criteria,  applied  the  60-40  technical  to  cost  factor,  and  ranked  the  offerors  as 
follows:  Chemonics,  Deloitte,  Carana,  Booz-Allen,  Abt,  and  Barents.  Barents  was  the 
only  offeror  in  the  top  six  that  did  not  receive  a  contract  award.  The  agency  said  the 
awards  were  not  based  solely  on  scores.  A  best  value  analysis  was  performed.  The 
agency  stated  that  it  initially  decided  to  make  awards  to  Chemonics,  Deloitte,  and  Carana 
because  they  were  in  the  top  six  technically  and  in  the  top  four  (lowest)  in  price.  After 
the  first  three  firms  were  ranked,  the  contracting  officer  ranked  Booz-Allen  fourth  overall 
with  Abt  finishing  fifth — ahead  of  Barents.  The  agency  noted  that  while  Barents 
technical  score  exceeded  Booz-Allen's  and  Abt's,  the  lower  prices  of  Booz-Allen  and  Abt 
offset  any  technical  advantage  enjoyed  by  the  protester.  The  agency  awarded  contracts  to 
the  five  firms  ranked  higher  than  Barents  plus  one  SBA  8(a)  company.  Barents  protested. 

Among  other  allegations,  Barents  argued  that  AID  failed  to  perform  a  cost  realism 
analysis  in  accordance  with  the  solicitation.  Barents  maintained  that  AID  was  required  to 
evaluate  whether  the  costs  proposed  by  each  offeror  were  consistent  with  the  firm's 
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technical  proposal.  The  RFP  also  stated  that  costs  could  have  been  adjusted  as  a  result  of 
a  cost  realism  analysis  to  evaluate  the  proposals  fairly.  Specifically,  Barents  asserted  that 
Booz-Allen  placed  a  contingency  in  their  proposal,  which  should  have  been  found 
unacceptable  by  the  agency.  Booz-Allen's  proposal  stated:  "Some  of  the  labor  hours 
included  in  this  cost  estimate  are  bid  at  off-site  rates,  substantially  lower  than  rates 
normally  applicable  to  services  performed  at  our  facility.  These  rates  are  based  upon  the 
provision  of  the  following  property /facilities  by  the  client  or  one  of  our  subcontractors. 
(a)  Office  facilities  (b)  Communications,  i.e.  local  and  long  distance  telephone  service 
(c)  Copying  facilities  (d)  Computer  facilities,  as  required  (e)  General  office  supplies 
(f)  parking  facilities.  The  costs  proposed  are  specifically  conditioned  upon  the 
availability  of  the  items  set  forth  above."  [Ref.  23]    Barents  argued  that  this  provision 
made  the  fixed  daily  salaries  and  multipliers  contingent  on  the  availability  of  the  listed 
items.  Therefore,  Booz-Allen  failed  to  comply  with  the  RFP  because  the  listed  items 
were  supposed  to  be  factored  into  the  salaries  and  multipliers.    Barents  also  maintained 
that  each  of  Booz-Allen's  proposed  multipliers  for  personnel  included  the  statement: 
"Assumes  bilateral  agreements  between  [AID]  and  participating  government  cover  any 
cost  incurred  in  respect  to  tax,  duties,  bonding,  and  any  social  welfare  costs."  [Ref.  23] 
Barents  stated  that  the  solicitation  did  not  outline  such  agreements.  The  protester  argued 
that  since  Booz-Allen's  daily  salaries  were  based  on  the  assumption  that  costs  would  be 
paid  by  AID  or  the  participating  Government,  Booz-Allen's  proposal  was  unacceptable. 


85 


AID  countered  by  stating  only  that  Booz-Allen's  proposal  was  in  conformity  with 
the  solicitation.  However,  the  agency  had  no  explanation  for  the  awardee's  statement  that 
the  proposed  costs  were  specifically  conditioned  upon  the  availability  of  the  listed  items 
that  were  supposed  to  already  be  included  in  the  fixed  daily  salaries  and  multipliers.  AID 
also  took  the  position  that  none  of  the  alleged  objectionable  language  from  Booz-Allen's 
proposal  was  included  in  the  final  contract,  so  it  had  no  impact  on  the  award.  GAO  found 
the  agency  evaluation  unreasonable  and  stated  that  it  was  clear  that  Booz-Allen's 
proposal  did  not  comply  with  the  solicitation  requirement  for  fixed  multipliers.  In 
conjunction,  AID's  cost  realism  analysis  of  the  awardee's  proposal  was  inadequate.  The 
Comptroller  General  also  addressed  AID's  assertion  that  the  contract  was  devoid  of 
objectionable  language;  thus,  the  proposal  flaws  were  of  no  matter.  GAO  stated  that  an 
awarded  contract  does  not  have  to  incorporate  all  aspects  of  a  proposal.  However,  the 
contract  may  not  vary  materially  from  the  offer.  GAO  found  that  the  contract  was 
materially  different.  In  addition,  GAO  held  that  the  conditions  placed  in  the  proposal  by 
Booz- Allen  removed  the  fixed  nature  (cap)  of  the  prices.  The  awardee  circumvented  the 
cap  in  this  case,  and  the  Government  was  no  longer  shielded  from  cost  growth.  In  such  a 
situation,  GAO  held  that  a  proper  cost  realism  analysis  was  required.  GAO  found  that  a 
proper  cost  realism  analysis  was  not  held  because  it  did  not  include  an  evaluation  of  the 
extent  to  which  Booz-Allen's  proposed  costs—represented  what  the  contract  should  cost. 
GAO  recommended  that  AID  clarify  the  RFP,  request  revised  BAFOs,  and  reevaluate. 
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2.  Geo-Centers,  Inc. 

Geo-Centers,  Inc.,  B-276033,  Comptroller  General  of  the  United  States,  5  May 
J  997,  (Geo)  is  a  protest  in  which  agency  failure  to  properly  conduct  price  and  cost 
realism  analyses  on  sample  task  costs  resulted  in  sustainment.  Geo  protested  the  Army's 
award  of  a  cost-plus-fixed-fee,  task  order  requirements  contract  for  scientific  and 
technical  support  services  for  the  Health  Effects  Research  Program  (HERP).  HERP  was 
part  of  the  Army's  Center  for  Health  Promotion  and  Preventative  Medicine  at  Aberdeen 
Proving  Ground,  Maryland.  The  contract  was  for  one  base  year  with  two  option  years. 
The  Army  determined  Dynamic  Corporation  (Dynamic)  to  be  the  offeror  whose  proposal 
represented  the  best  value  to  the  Government. 

The  RFP  outlined  five  evaluation  factors:  technical  merit,  management  merit, 
performance  risk,  cost,  and  subcontracting  plan.  The  technical  merit  and  management 
merit  were  to  be  combined  and  scored  numerically  on  a  1 ,000  point  scale.  Performance 
risk  was  to  be  evaluated  on  a  narrative  basis.  The  subcontracting  plan  was  to  be  graded 
on  a  go/no  go  (pass  or  fail)  basis.  The  solicitation  stated  that  technical  merit  and 
management  merit  were  significantly  more  important  than  cost,  and  cost  was  slightly 
more  important  that  performance  risk.  Cost  was  to  assume  greater  importance  in  the 
event  of  two  or  more  proposals  being  evaluated  as  equal  in  the  technical  merit  and 
management  merit  criteria.  The  RFP  contained  three  sample  tasks  for  which  each  offeror 
was  to  submit  technical  responses  with  sample  task  cost  proposals.  These  sample  task 
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cost  proposals  were  in  addition  to  the  total  offeror  cost  proposal  required  by  the 
solicitation. 

The  sample  task  cost  proposals  were  to  be  evaluated  for  price  reasonableness  and 
cost  realism,  and  the  solicitation  defined  both  terms.  Cost  realism  was  defined  as 
"whether  or  not  the  offeror  has  proposed  sufficient  resources  to  successfully  perform  the 
contract  and  sample  task  work."  Price  realism  was  outlined  as  "a  matter  of  the 
competitiveness  of  the  offeror's  proposal,  considering,  the  price  and  the  merit  factors 
area."  [Ref.  25]  Geo  received  initial  proposal,  post  discussion,  and  BAFO  scores  of  930, 
998,  and  1000  respectively  (1000  point  scale).  Dynamic  received  scores  of  680,  775,  and 
1000.  Both  proposals  were  found  to  be  technically  equal,  so  the  Army  awarded  the 
contract  to  the  offeror  with  the  lowest  overall  proposed  cost — Dynamic.  Geo  protested. 

Geo  argued  that  the  agency  failed  to  conduct  cost  realism  or  price  realism 
evaluations  on  the  sample  task  costs  in  Dynamic's  BAFO.  The  protester's  argument 
pointed  to  the  fact  that  Dynamic  increased  labor  hours  and  costs  for  the  three  sample 
tasks,  yet  they  lowered  their  overall  proposal  costs  by  25  percent.  Geo  argued  that  these 
major  cost  shifts  should  have  drawn  reevaluations  of  price  and  cost  reasonableness  for  the 
sample  tasks  from  the  Army.  The  agency  maintained  that  the  pricing  in  the  sample  task 
was  not  intended  as  a  tool  for  establishing  a  basis  for  price  reasonableness.    The  RFP 
supported  the  agency's  assertion  stating  that  the  sample  tasks  were  not  used  to  determine 
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the  total  contract  price;  rather,  they  were  merely  a  gauge  for  contractor  understanding  of 
the  effort  required  for  contract  completion. 

However,  GAO  stated  that  by  using  a  task  order  requirements  contract,  the  agency 
acknowledged  that  it  was  not  able  to  determine  the  exact  quantity  of  services  and 
materials  needed  during  the  performance  period.  Therefore,  the  cost  to  the  Government 
would  have  varied  based  on  the  actual  task  orders  issued,  and  most  likely,  with  a 
contractor's  efficiency  in  performing  the  issued  tasks.  The  offerors'  overall  proposal 
costs  were  not  dependable.  GAO  stated  that  the  sample  task  cost  proposals  were  a 
significant  indicator  of  future  cost  to  the  Government.  In  addition,  the  solicitation 
advised  that  sample  task  costs  would  be  evaluated  for  both  cost  realism  and  price 
reasonableness,  and  the  agency  was  bound  to  use  the  evaluation  tools  identified  in  the 
RFP.  GAO  determined  that  the  Army  failed  to  conduct  price  and  cost  realism  evaluations 
on  Dynamic's  BAFO.  The  failure  to  analyze  the  cost  realism  of  the  proposed  sample 
tasks  left  the  agency  open  to  the  possibility  that  Dynamic  did  not  accurately  account  for 
all  of  the  costs  that  will  be  incurred  during  the  performance.  GAO  further  determined 
that  the  Army  failed  to  conduct  any  price  realism  analyses  of  sample  task  costs  (in 
proposals  or  BAFOs)  despite  the  requirement  in  the  solicitation.  GAO  found  Geo  had 
been  prejudiced  because  the  Comptroller  General  could  not  conclude  that  the  protester 
would  not  have  had  a  substantial  chance  of  award  had  the  agency  properly  analyzed  the 
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sample  task  costs.  As  a  result,  GAO  sustained  the  protest  and  recommended 
reevaluation.  [Ref.  25] 

3.  Arora  Group,  Inc. 

Matter  of  Arora  Group,  Inc.,  B-277674,  Comptroller  General  of  the  United 
States,  10  November  1997,  (Arora)  is  a  protest  in  which  the  Federal  agency  unreasonably 
determined  that  the  protester's  proposal  lacked  price  realism.  Arora  Group  protested  the 
U.S.  Navy's  award  of  a  fixed-price  contract  for  the  services  often  pharmacists  for  the 
National  Naval  Medical  Center  in  Bethesda,  Maryland.  The  contract  was  to  be  a  small 
business  set  aside  under  the  SBA's  8(a)  program.  The  solicitation  sought  the  services  of 
ten  full-time  and  five  optional  pharmacists  for  a  base  period  of  four  option  periods 
spanning  a  maximum  of  five  years.  The  Navy  determined  Saratoga  Medical  Center,  Inc. 
(SMC)  to  be  the  best  value  offeror. 

The  RFP  stated  that  the  best  value  award  would  be  made  to  the  offeror  whose 
proposal  conformed  to  the  specified  minimum  healthcare  worker  qualifications  and 
offered  the  Government  with  the  best  combination  of  past  performance  and  price. 
Offerors  had  to  provide  proof  that  they  would  provide  at  least  ten  individual  healthcare 
workers.  Past  performance  was  outlined  as  significantly  more  important  than  price.  In 
the  event  two  or  more  offerors  had  equal  past  performance,  price  would  be  the  deciding 
factor.  The  solicitation  also  stated  that  price  realism  analyses  would  be  conducted  to 
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preclude  unrealistically  low  cost  estimates.  These  analyses  were  to  be  conducted  within 
the  context  of  the  offerors'  past  performance. 

The  agency  received  eight  proposals.  Both  the  protester  and  awardee  earned  past 
performance  grades  of  Good.  With  respect  to  price  realism,  the  Federal  Acquisition 
Circular  (FAC  90-23)  recommended  an  annual  wage  escalation  rate  of  three  percent.  The 
Navy  had  indicated  that  provisions  for  wage  escalation  should  be  included  in  each 
offeror's  proposal  to  guard  against  labor  turnover  and  wage  increase.  The  protester  failed 
to  propose  wage  escalation  during  the  option  periods  covered  in  the  RFP.  The  Navy 
determined  that  the  protester's  failure  to  provide  for  any  salary  escalation  presented  a 
price  realism  issue  because  it  created  risk  that  the  offeror  might  not  be  able  to  maintain  a 
pool  of  qualified  personnel  or  recruit  suitable  replacements.  The  protester  was  advised  of 
this  perceived  shortcoming  in  its  proposal  through  discussions  but  did  not  submit  wage 
escalation  data  in  its  revised  proposal.  The  agency  determined  SMC  to  be  the  best  value 
offeror  and  awarded  them  the  contract.  Arora  Group  protested. 

The  protester  argued  that  the  Navy's  price  realism  analysis  was  not  in  accordance 
with  the  RFP  and  plainly  erroneous.  The  protester  asserted  that  the  agency  incorrectly 
conducted  price  realism  analysis  because  they  failed  to  consider  the  fact  that  Arora  Group 
addressed  the  recruitment/retention  issue  through  long-term  employment  agreements.  In 
addition,  the  protester  had  planned  for  recruitment  costs  in  its  General  and  Administrative 
(G&A)  budget  in  the  event  of  personnel  turnover.  The  Navy  countered  that  the  protester's 
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plan  did  not  provide  convincing  evidence  that  they  would  be  able  to  retain  the  healthcare 
workers  through  the  five-year  span  of  the  contract  without  wage  escalation.  The  agency 
also  stated  that  the  protester's  G&A  budget  for  recruitment  was  vague. 

GAO  held  that  the  agency's  decision  to  reject  Arora's  proposal  because  its  failure 
to  propose  wage  escalation  created  an  unacceptable  price  realism  risk  was  neither 
reasonable  nor  consistent  with  the  evaluation  criteria.  The  Comptroller  General  stated 
that  the  agency  failed  to  adequately  evaluate  the  information  presented  by  the  protester. 
With  respect  to  the  employment  contracts,  the  Navy  had  requested  copies  of  the  labor 
agreements  during  discussions.  The  protester  failed  to  provide  these  agreements; 
therefore,  the  Navy  essentially  ignored  the  possible  impact  of  this  approach.  GAO  found 
this  action  to  be  unreasonable.  GAO  stated  that  while  copies  of  labor  agreements  may 
have  provided  reassurance  to  the  Navy,  Arora's  failure  to  provide  the  copies  did  not  mean 
the  agency  could  completely  discount  the  protester's  efforts  in  this  regard.  Furthermore, 
the  agency  improperly  evaluated  the  protester's  G&A  budget  for  recruitment  and 
understated  its  value  by  90  percent.  GAO  held  that  the  combination  of  the  unreasonable 
discounting  of  the  labor  agreements  and  the  erroneous  evaluation  of  the  G&A  budget 
prejudiced  the  protester.  The  Comptroller  General  sustained  the  protest.  [Ref.  32] 
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E.         AGENCY  FAILURE  TO  FOLLOW  THE  SOLICITATION 

1.  Arora  Group,  Inc. 

Arora  also  demonstrated  agency  failure  to  follow  the  solicitation.  The  RFP  stated 
that  the  price  realism  analyses  were  to  be  conducted  within  the  context  of  the  offerors' 
past  performance.  Both  the  protester  and  awardee  received  past  performance  scores  of 
Good.  The  protester  had  offered  a  lower  price  than  the  awardee.  However,  the  agency 
ignored  the  evaluation  criteria  and  rejected  Arora  Group's  proposal  due  to  concern  about 
price  realism  (wage  escalation)  risk.  With  respect  to  the  protester's  proposal,  GAO  held 
that  the  Navy  failed  to  conduct  price  realism  analysis  and  tradeoff  as  outlined  in  the  RFP. 
The  agency  had  improperly  made  price  realism  a  separate  and  determinative  evaluation 
factor.  GAO  sustained  the  protest  and  recommended  reconsideration  of  the  evaluation 
criteria,  BAFOs,  and  reevaluation.    [Ref.  32] 

2.  Tri-State  Government  Services,  Inc. 

Tri-State  Government  Services,  Inc.,  B-277315.2,  Comptroller  General  of  the 
United  States,  15  October  1997,  {Tri-State)  is  a  protest  in  which  agency  failure  to  hold 
the  awardee  to  the  terms  of  the  RFP  combined  with  improper  post-B  AFO  discussions 
resulted  in  sustainment.  Tri-State  protested  the  Defense  Logistics  Agency  (DLA)  award 
of  a  fixed  firm  price  contract  for  the  removal,  transportation,  and  disposal  of  hazardous 
waste  items  located  at  various  Defense  Reutilization  and  Marketing  Sites  (DRMS).  The 
solicitation  anticipated  an  1 8-month  base  period  with  two  option  years.  DLA  found 
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Associated  Environmental  Services,  Inc.  (AES)  to  be  the  offeror  whose  proposal 
represented  the  best  value  to  the  Government. 

The  RFP  required  separate  technical,  price,  and  past  performance  proposals  and 
warned  offerors  that  proposals,  which  did  not  include  all  required  information  in  the 
prescribed  format,  could  be  excluded.  The  solicitation  outlined  price,  past  performance, 
socioeconomic  plan,  and  Mentoring  Business  Agreement  (MBA)  participation  as 
evaluation  factors.  Past  performance  and  price  were  outlined  as  equal  in  importance; 
furthermore,  they  were  also  stated  to  be  significantly  more  important  than  the  value  of 
socioeconomic  plan  and  MBA  combined.    The  REP  divided  the  work  to  be  performed  in 
to  94  contract  line  items  (CLIN)  for  different  waste  substances.  Each  proposal  was  to 
provide  a  single  unit  price  for  each  CLIN  in  a  space  provided  in  the  solicitation  price 
schedule.  The  single  unit  prices  were  to  be  multiplied  by  the  agency's  estimated  quantity 
(of  hazardous  waste)  to  arrive  at  an  extended  price  for  each  CLIN,  and  then  all  CLIN 
extended  prices  would  be  summed  to  reveal  a  total  price.    Eleven  offerors  submitted 
proposals  with  only  five  surviving  the  competitive  range  determination.  Four  of  the  five 
proposals  (including  AES  and  Tri-State)  were  graded  as  good  under  both  past 
performance  and  a  best  value  composite  rating.  Price  became  the  determining  factor. 
AES  ranked  first  in  price  with  a  total  of  $  3.29  million.  Tri-state  finished  second  in  price 
with  an  offer  totaling  $  3.44  million.  AES  was  awarded  the  contract  and  Tri-state 
protested. 
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Tri-state  argued  that  AES  improperly  modified  the  agency's  price  schedule  and 
should  not  have  been  considered.  Instead  of  submitting  single  unit  prices  for  each  CLIN, 
AES  split  DLA's  estimated  quantities  listed  on  the  price  schedule  and  submitted  one 
price  for  a  number  that  it  described  as  first  quantities  ordered  and  a  different  price  for 
what  it  designated  as  next  quantities  ordered.  In  each  of  the  split  CLINs,  the  prices  were 
higher  for  the  first  quantities  ordered  and  lower  for  the  next  quantities.  The  protester 
contended  that  the  modified  pricing  scheme  gave  AES  an  unfair  advantage  because  such 
a  pricing  method  was  not  available  to  all  offerors.  Tri-state  also  asserted  that  DLA 
conducted  improper  post-BAFO  discussions  with  AES.  Tri-state  maintained  that  the 
agency  re-opened  discussion  with  the  awardee  to  negotiate  prices  for  any  quantities 
ordered  beyond  the  Government  estimates  shown  in  the  RFP. 

Regarding  the  pricing  scheme,  the  agency  countered  claiming  that  the  RFP  did  not 
prohibit  AES's  pricing  strategy.  DLA  considered  the  awardee 's  pricing  strategy  as  only  a 
minor  irregularity  and  consistent  with  the  RFP.  GAO  held  that  the  agency  improperly 
accepted  the  awardee's  offer  because  it  did  not  conform  to  the  material  terms  and 
conditions  of  the  solicitation.  GAO  stated  that  an  irregularity  in  an  offer  that  benefits  the 
offeror  must  be  extended  to  all  firms  submitting  proposals.  GAO  continued  by  stating 
that  if  the  beneficial  irregularity  is  not  extended  to  all  offerors,  the  proposal  containing 
such  an  irregularity  may  not  be  accepted.  In  determining  prejudice,  GAO  found  that  if 
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Tri-State  had  been  allowed  to  use  AES's  pricing  method,  the  protester  would  have  had  a 
lower  overall  price  than  AES  by  $  130,000. 

With  respect  to  Tri-State's  assertion  of  improper  discussions  with  the  awardee,  the 
agency  asserted  that  the  contracting  officer  believed  that  it  was  reasonably  clear  from 
their  proposal  that  the  prices  AES  had  quoted  for  next  quantities  would  apply  to  future 
quantities  beyond  the  Government  estimates.  DLI  claimed  that  they  only  conducted 
clarifications  with  AES  to  clear  the  ambiguity  as  to  the  price  of  these  follow-on 
quantities.  GAO  held  that  the  exchanges  were  actually  discussions  for  two  reasons. 
First,  the  Comptroller  General  held  that  it  was  not  reasonably  clear  that  the  next  quantity 
prices  would  apply  beyond  the  Government  estimates.  Second,  the  information 
exchanged  was  necessary  to  determine  the  acceptability  of  AES's  proposal.  GAO  held 
that  Tri-State  was  prejudiced  because  of  the  improper  pricing  and  discussions.  GAO 
sustained  the  protest  with  a  recommendation  to  terminate  the  AES  contract  and  award  to 
Tri-State.  [Ref.  27] 
F.         ANALYSIS  AND  CONCLUSION 

The  purpose  of  this  analysis  is  to  provide  contracting  officers  with  insight  as  to 
possible  agency  miscues  when  evaluating  cost  or  price. 

1.  Evaluate  the  Total  Price  or  Cost 

Failure  to  evaluate  the  total  price  or  cost  is  an  agency  error  that  must  not  be 
committed.  However,  Boeing  showed  that  agency  efforts  to  evaluate  cost  risk  cannot  be 
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executed  to  the  exclusion  of  a  total  cost  comparison  between  offerors.  The  agency 
(SOCOMj  seemed  to  be  unfamiliar  with  this  requirement  even  though  their  RIP  clearly 
mandated  a  total  cost  comparison.   ( JAO  cited  a  general  rule  of  law  originating  with 
CICA  09H4)  that  required  inclusion  of  cost  or  price  as  a  significant  factor  in  the 
evaluation  of  proposals.    The  implication  for  contracting  officers  is  clear,  failure  to 
consider  total  cost  is  a  very  wirmable  protest  for  firms    (  ontracting  officers  cannot  gel  so 
caught  up  with  slick  cost  analysis  tools  that  they  make  tradeoffs  without  consideration  of 
a  basic  total  COSt/price  comparison.   Boeing  also  demonstrated  an  erroneous  agent 
adjustment  of  labor  cost  coupled  with  a  failure  to  inform  the  offeror  through  meaningful 
discussions  can  contribute  to  a  sustainment.  Although  the  protester  did  not  provide  a 
summation  of  its  labor  cost  figures  as  was  required  by  the  RFP,  SOCOM  clearly  erred  by 
(]j  failing  to  discuss  the  apparent  mistake  with  the  offeror  and  (2)  erroneously  adjusting 
the  labor  cost  upward.  GAO  stated  the  agency's  error  regarding  the  labor  cost  tvaj  not 
nearly  as  critical  as  their  failure  to  inform  the  -  during  discussions    GAO'fl  holding 

on  this  point  was  founded  in  the  FAR  requirement  (pre  and  post-Part  1 
agencies  to  ensure  that  discussions  (when  held;  are  meaningful.  [Kef.  13:  15.601  J    for 
SOCOM's  discussions  to  be  meaningful.,  they  would  have  had  to  inform  Boeing  of  the 
aspects  of  their  proposal  that  had  to  be  addressed  in  order  for  the  offeror  to  have  a 

able  chance  to  obtain  award.  SO(  OM  had  to  discuss  the  perceived  mistake  in 
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labor  cost.  The  implication  for  contracting  officers  is  that  they  must  ensure  discussions 
convey  any  proposal  shortcomings  for  the  discussions  to  be  meaningful.  [Ref.  24] 

Sylvest,  like  Boeing  shows  that  an  action  that  indirectly  undermines  the  total  cost 
or  price  comparison  can  result  in  a  sustained  protest.  The  agency  (TV A)  failed  to 
compare  adequately  proposal  prices  by  awarding  the  contract  to  an  offeror  who  had  made 
a  necessary  hard  disk  modification  (memory  increase)  an  option  in  their  proposal.  GAO 
held  that  the  agency  did  not  consider  the  optional  price  increase  when  they  compared 
proposal  prices  resulting  in  a  lower  proposed  price  by  the  awardee.  In  other  words,  TVA 
did  not  compare  the  awardee 's  total  price  (hard  disk  plus  increased  memory)  to  the 
protester's  total  price.  GAO's  holding  looked  to  the  FAR  requirement  for  acceptable 
proposals  to  be  responsive.  [Ref.  13:  9.104]  The  awardee' s  proposal  was  not  responsive. 
The  implication  for  contracting  officers  is  that  they  must  be  wary  of  proposals  that  vary 
from  solicitations.  If  the  variance  is  material,  the  proposal  is  unacceptable.  [Ref.  26] 
2.  Thoroughly  Conduct  Cost/Price  Realism  Analyses  When  Required 

Failure  to  conduct  cost/price  realism  analyses  undermines  price  evaluation. 
Barents  demonstrates  that  improper  cost  realism  can  result  in  sustainment.  The 
solicitation  required  that  all  costs  be  included  into  fixed  multipliers  so  that  the  agency 
could  evaluate  an  averaged  burdened  daily  rate  in  pursuit  of  best  value.  The  agency's 
(AID)  problem  stemmed  from  an  awardee  that  placed  a  contingency  on  their  price 
multipliers.  The  multiplier  rate  was  contingent  on  the  availability  of  certain  property  and 
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facilities.  The  RFP  stated  that  multipliers  could  have  been  adjusted  through  a  cost 
realism  analysis  to  evaluate  the  proposals  fairly.  A  proper  cost  realism  analysis  would 
have  uncovered  the  discrepancy  (contingency)  in  the  awardee's  proposal.  GAO  held  that 
the  cost  realism  analysis  was  inadequate.  GAO's  holding  was  founded  in  the  requirement 
that  a  contract  may  not  materially  vary  from  a  proposal.  The  Comptroller  General 
determined  that  the  contract  did  materially  vary  from  the  offer,  and  the  cost  realism 
analysis  did  not  shed  light  on  the  unacceptable  contingency  levied  by  the  awardee.  The 
implication  for  contracting  officers  is  that  they  must  make  use  of  tools  such  as  cost/price 
realism  analyses.  These  tools  guard  against  proposals  that  materially  differ  from 
solicitations  which  often  leads  to  contracts  that  materially  differ  from  the  proposals  as  in 
this  case.  [Ref.  23] 

Geo  showed  how  the  agency's  failure  to  conduct  both  cost  and  price  realism 
analyses  resulted  in  a  sustained  protest.  The  awardee's  BAFO  increase  in  sample  task 
costs,  combined  with  a  25  percent  decrease  in  overall  proposal  costs,  should  have  drawn  a 
cost  and  price  reevaluation  by  the  agency.  It  did  not.  Sample  tasks  and  associated  cost 
and  price  realism  analyses  were  required  by  a  solicitation  for  this  task  order  requirements 
contract.  The  analyses  of  the  sample  tasks  were  identified  by  the  RFP  as  only  measures 
of  contractor  understanding.  However,  GAO  stated  that  the  sample  task  costs  were  the 
only  reasonable  method  for  cost  estimation  because  the  cost  to  the  Government  would 
have  varied  based  on  the  issuance  of  actual  task  orders.  In  addition,  GAO  found  that  no 
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price  realism  analyses  had  been  conducted  on  any  of  the  offerors'  proposals  or  BAFOs. 
GAO's  holding  arose  from  a  previous  protest  (Group  Technologies  Corp.,  B-240736,  19 
December  1990).    The  Comptroller  General  had  held  that  in  task  order  contracts,  the  use 
of  sample  tasks  permitted  agencies  to  review  the  approaches  and  efficiencies  of  different 
contractors  under  simulated  contract  conditions  which  require  the  submission  of  detailed 
cost  proposals.  [Ref.  25]  In  Geo,  where  the  solicitation  advised  that  sample  tasks  would 
undergo  cost  and  price  realism  analyses,  the  agency  was  bound  to  use  the  evaluation 
methods  identified  in  the  solicitation.  The  implication  for  contracting  officers  lies  in  the 
need  for  Federal  agencies  to  use  the  evaluation  tools  indicated  in  their  solicitations. 
Failure  in  this  regard  provides  a  strong  basis  for  sustainment.  [Ref.  25] 

Arora  demonstrates  how  improper  and  inaccurate  price  realism  analyses  can  result 
in  sustainment.  The  Navy  failed  to  account  for  the  protester's  plan  to  counter  wage 
escalation  through  long-term  labor  agreements.  In  addition,  the  agency  erroneously 
assessed  the  protester's  G&A  budget  for  recruitment  (in  the  event  of  labor  turnover). 
GAO  held  that  the  evaluation  of  the  protester's  proposal  was  unreasonable  and 
inconsistent.  This  holding  stemmed  from  an  inherent  requirement  for  the  Government  to 
provide  accurate  cost  realism  analyses  when  such  analyses  are  outlined  in  the  solicitation. 
The  implication  for  contracting  officers  is  that  great  care  must  be  taken  to  ensure  accurate 
and  reasonable  price  analyses.  [Ref.  32] 
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3.  Follow  the  Solicitation 

As  in  previous  chapters,  a  Federal  agency's  inability  or  unwillingness  to  follow 
the  terms  of  solicitations  results  in  sustainment.  In  Arora,  the  RFP  stated  that  price 
realism  analyses  would  be  conducted  within  the  context  of  the  offerors'  past  performance. 
With  respect  to  the  protester,  the  Navy  ignored  the  requirement  to  analyze  price  realism 
within  past  performance  because  the  agency  had  concerns  about  the  risk  of  wage 
escalation  and  labor  turnover.  GAO  held  that  such  concerns  were  valid;  however,  the 
agency  was  still  required  to  adhere  to  the  solicitation  by  evaluating  price  realism  within 
the  context  of  past  performance.  The  Comptroller  General  bolstered  this  decision  by 
stating  that  an  agency  could  not  arbitrarily  make  price  realism  a  separate  and 
determinative  evaluation  factor  because  the  solicitation  clearly  stated  that  price 
reasonableness  would  be  evaluate  within  the  past  performance  factor.    The  implication 
for  the  contracting  officer  is  that  the  solicitation  must  be  followed  regardless  of  the 
unusual  aspects  of  an  offerors  proposal.  Fair  and  consistent  evaluation  cannot  be 
achieved  without  adherence  to  the  solicitation.  [Ref.  32] 

In  Tri-State,  the  awardee  gained  a  price  advantage  by  inappropriately  splitting  the 
CLINs  and  offering  both  first  quantity  and  next  quantity  prices.  GAO  held  that  this 
irregularity  benefited  the  awardee;  therefore,  the  protester  was  prejudiced.  GAO's  ruling 
stemmed  from  another  previous  protest  (Multi-Spec  Prods.  Group  Corp.,  B-245156.2,  11 
February  1992)  which  stated  when  an  irregularity  in  an  offer  results  in  benefits  to  an 
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offeror,  it  must  be  extended  to  all  offerors.  None  of  the  other  offerors  were  extended  the 
opportunity  to  make  use  of  the  awardee's  pricing  scheme.  GAO  determined  that  the 
awardee  had  an  unfair  advantage  and  sustained  the  protest.  The  implication  for 
contracting  officers  is  that  the  RFP  simply  must  be  followed.  In  addition,  improper 
discussions  were  held  regarding  the  pricing  of  quantities  beyond  Government  hazardous 
waste  estimates.  GAO  held  that  the  agency's  perceived  clarifications  were  actually 
discussions  because  they  involved  information  essential  to  determine  the  acceptability  of 
a  proposal.  The  rule  applied  by  GAO  was  that  if  discussions  are  held  with  one  offeror, 
they  must  be  held  with  all  offerors  in  the  competitive  range.  [Ref.  13:  15.601] 
Contracting  officers  must  be  cognizant  of  their  exchanges  with  contractors  and  ensure 
that  when  discussions  are  held,  all  offerors  must  be  included. 

Contracting  officer  should  take  these  valuable  lessons  with  them  when  evaluating 
cost/price.  First,  ensure  total  price  or  cost  is  evaluated  and  a  comparison  is  held  between 
all  offerors.  Second,  execute  price  and/or  cost  realism  analyses  when  required.  Finally, 
follow  the  solicitation.  Chapter  VII  examines  protests  that  were  sustained  due  to 
improper  agency  evaluation  of  labor  qualifications. 
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VII.      LABOR  QUALIFICATIONS 

A.  INTRODUCTION 

Improper  Federal  agency  evaluation  of  labor  qualifications  is  cited  as  a  sustaining 
argument  in  nine  percent  of  the  successful  protests.  Labor  qualifications  are  often 
important  elements  in  a  best  value  tradeoff.  Clearly,  a  special  qualification  brings  value 
to  an  offeror's  proposal  as  well  as  associated  costs.  These  valued  qualifications  must  be 
traded  off,  and  through  the  contracting  officer's  judgment,  best  value  is  determined.  The 
two  protests  that  were  sustained  due  to  improper  evaluation  of  labor  qualifications  are 
Barents  Group,  L.L.C.  and  For  Your  Information,  Inc.  This  chapter  thoroughly  describes 
the  two  sustained  protests  and  provides  analysis  and  conclusions  regarding  agency 
evaluation  of  labor  qualifications. 

B.  PROTESTS 

1.  Barents  Group,  L.L.C. 

Barents  Group,  L.L.C.;  B-276082,  B276082.2;  Comptroller  General  of  the  United 
States;  9  May  1997,  (Barents)  was  discussed  in  Chapter  VI  because  improper  agency 
evaluation  of  cost/price  is  a  sustaining  argument  in  this  protest.  Improper  agency 
evaluation  of  labor  qualifications  is  also  a  sustaining  argument.  Barents  protested  the 
award  of  an  Agency  for  International  Development  (AID)  contract  for  technical  expertise 
in:  (1)  economic  and  institutional  analysis  and  (2)  private  sector  development  issues.  The 
RFP  outlined  the  award  of  contracts  in  two  functional  areas;  however,  protests  were 
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issued  regarding  only  line  item  0001 — privatization  issues.  Each  proposal  was  to  be 
assigned  a  score  for  technical  and  cost.  Technical  was  weighted  60  percent  and  cost  40 
percent.  Technical  factors  included  the  following  (25  points  each):  (1)  personnel 
qualifications  and  experience,  (2)  quality  and  responsiveness,  (3)  demonstrated  corporate 
experience,  and  (4)  past  performance.  AID  found  Chemonics  International,  Inc. 
(Chemonics);  Booz-Allen  &  Hamilton,  Inc.  (Booz-Allen);  Abt  Associates,  Inc.  (Abt); 
Carana  Corporation  (Carana);  and  Deloitte  Touche  Tohmatsu  (Deloitte)  to  be  the  offerors 
who  submitted  proposals  representing  the  greatest  value  to  the  Government. 

Barents  argued  that  the  proposals  submitted  by  Chemonics  and  Carana  did  not 
comply  with  the  solicitation.  The  RFP  required  the  submission  of  three  resumes  for 
investment  bankers  to  be  assigned  to  the  project.  The  protester  maintained  that  in  both  of 
the  awardees'  proposals,  one  of  the  three  resumes  did  not  meet  the  qualification  of  an 
investment  banker.  Barents  asserted  that  investment  bankers,  by  definition,  underwrite 
and  sell  new  securities.  The  protester  pointed  out  that  Chemonics  and  Carana  had  both 
proposed  personnel  that  were  mere  consultants  and  not  investment  bankers.  The  protester 
went  so  far  as  to  cite  the  Dictionary  of  Banking  (Jerry  M.  Rosenberg  (1983))  which  stated 
that  an  investment  bank  "serves  as  an  underwriter  for  new  issues  of  bonds  or  stocks  and 
as  part  of  a  syndicate,  redistributes  the  issues  to  investors."  An  investment  banker  was 
defined  as  "the  middleman  between  the  corporation  issuing  new  securities  and  the 
public."  [Ref.  23] 
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AID  countered  by  arguing  that  the  RFP  did  not  specify  the  exact  qualifications 
required  for  the  investment  bankers.  Instead,  each  offeror  was  to  determine  the 
competencies  necessary  to  complete  the  required  work.    GAO  agreed  that  the  RFP  did 
not  contain  definitions  of  any  of  the  various  required  labor  categories;  however,  the 
solicitation  did  ask  for  separate  sets  of  resumes  for  each  of  the  labor  categories  including: 
corporate  financial  advisor,  commercial  banker,  and  investment  banker.  GAO  held  that 
the  only  reasonable  interpretation  of  the  RFP  was  that  the  resumes  for  the  various  labor 
categories  should  have  represented  the  differences  between  the  categories.  In  short,  the 
terms  of  the  solicitation  were  not  met  because  inexperienced  consultants  did  not  qualify 
as  investment  bankers.  GAO  determined  that  the  combination  of  the  improper 
evaluations  of  price  (Chapter  IV)  and  labor  qualifications  did  prejudice  Barents.  The 
Comptroller  General  recommended  that  AID  clarify  the  RFP,  request  revised  BAFOs, 
and  reevaluate.  [Ref.  23] 

2.  For  Your  Information,  Inc. 

For  Your  Information,  Inc.,  B-278352,  Comptroller  General  of  the  United  States, 
15  December  1997,  (FYT)  is  another  protest  where  improper  agency  evaluation  of 
personnel  qualifications  resulted  in  sustainment.  FYI  protested  the  U.S.  Coast  Guard 
(USCG)  award  of  a  fixed-price,  indefinite  delivery  /indefinite  quantity  contract  for 
information  technology  support  services.  The  contract  was  restricted  under  the  Small 
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Business  Administration's  8(a)  program.  USCG  found  RGII  Technologies,  Inc.  (RGII) 
to  be  the  offeror  representing  best  value  to  the  Government. 

The  RFP  called  for  a  technical  evaluation  team  (TET)  to  evaluate  the  proposals  in 
two  phases.  Under  Phase  I,  the  TET  would  evaluate  mini-proposals  and  past 
performance.  Phase  II  would  be  evaluated  under  oral  presentation/slides,  personnel  data 
forms  (PDF),  and  cost/price.  The  proposals  were  to  be  evaluated  under  the  following 
color/adjectival  system:  blue/superior,  green/satisfactory,  yellow/marginal,  and 
red/unsatisfactory.  Fifty-one  proposals  were  received.  Offerors  receiving  blue  ratings  in 
Phase  I  were  invited  to  continue  to  Phase  II.  Offerors  not  receiving  blue  ratings  could 
continue  if  they  chose  to  do  so.  Both  FYI  and  RGII  received  green  ratings  in  Phase  I  and 
elected  to  continue  into  Phase  II  evaluations.  Phase  II  was  described  to  be  more  important 
than  Phase  I.  The  solicitation  identified  ten  Phase  II  labor  categories  as  key  personnel 
positions.  Offerors  were  required  by  the  RFP  to  provide  PDFs  (containing  qualifications, 
education,  and  employment  history)  for  50  percent  of  the  proposed  personnel  in  each 
labor  category. 

All  BAFO's  received  by  offerors  for  Phase  II  evaluation  were  determined  to  have 
overall  ratings  of  green  and  were  found  to  have  reasonable  and  realistic  prices.  The  factor 
in  question  in  this  protest  was  personnel  requirements.  Only  FYI's  BAFO  was  found 
satisfactory  (green)  under  personnel  requirements.  RGII's  BAFO  was  rated  as 
yellow/marginal  because  PDF's  (for  50  percent  of  the  proposed  personnel)  were  not 
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submitted  for  five  of  the  ten  labor  categories.  The  TET  recommended  F  YI  for  award  to 
the  source  selection  official  (SSO)  even  though  FYI's  price  was  higher.  The  SSO  asked 
for  justification  as  to  why  the  contract  recommendation  was  being  made  on  a  basis  other 
than  lowest  price.  The  TET  responded  stating  that  it  doubted  RGII's  ability  to  obtain  the 
qualified  personnel  because  they  failed  to  propose  qualified  workers  even  after  the  matter 
was  addressed  during  discussions.  The  SSO  requested  information  on  RGITs  past 
performance  and  was  told  it  was  superior.  The  SSO  found  that  RGII's  offer  was  the  best 
value  for  the  Government  because  the  risk  associated  with  the  lack  of  qualified  personnel 
was  outweighed  by:  (1)  past  performance  and  (2)  the  fact  that  PDF  risk  was  to  be 
minimized  by  an  RFP  provision  for  USCG  approval  of  all  PDFs  before  granting  access  to 
the  work  site.  The  SSO  also  determined  that  any  proposal  shortcomings  regarding 
personnel  qualifications  would  be  dealt  with  under  applicable  labor  substitution  clauses 
during  contract  administration.  The  SSO  stated  that  the  cost  differential  did  not  justify 
granting  FYI  the  contract  award.  RGII  was  awarded  the  contract,  and  FYI  protested. 

FYI  argued  that  the  awardee's  proposal  should  have  been  found  unacceptable  by 
the  agency.  GAO  concurred  pointing  to  the  RFP  that  stated  that  the  quality  and  quantity 
of  all  proposed  personnel  had  to  meet  the  minimum  qualifications  stated  in  the 
solicitation.  GAO  asserted  that  a  proposal  that  fails  to  conform  to  the  material  terms  and 
conditions  of  a  solicitation  must  be  considered  unacceptable.  The  Comptroller  General 
stated  that  there  was  no  question  that  personnel  qualifications  were  material  terms  of  the 
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RFP  and  held  that  RGII's  proposal  did  not  satisfy  the  personnel  qualification 
requirements.  GAO  also  stated  that  qualification  problems  could  not  be  handled  during 
contract  administration  because  the  RFP  required  the  contractor  to  assign  the  personnel 
named  in  the  PDFs  to  the  contract.  Therefore,  the  only  agency  opportunity  to  approve  or 
disapprove  proposed  individuals  was  during  the  evaluation — regardless  of  any  clauses 
attached  to  the  contract  by  the  contracting  officer.  Thus,  the  agency's  post-award 
approval/disapproval  rights  were  eliminated  by  this  statement  in  the  proposal.  The 
Comptroller  General  found  that  FYI  had  been  prejudiced.  GAO  recommended 
termination  of  RGII's  contract  and  award  of  the  contract  to  FYI.  [Ref.  28] 
C.         ANALYSIS  AND  CONCLUSION 

The  purpose  of  this  analysis  is  to  provide  contracting  officers  with  insight  as  to 
possible  agency  miscues  when  evaluating  labor  qualifications. 

1.  Certainty  of  Requirements  is  Critical  to  Successful  Evaluation  and 

Award 

Barents  is  a  fascinating  protest  because  the  outcome  hinged  on  the  definition  of  a 
professional  title.  The  RFP  called  for  three  Investment  Banker  resumes  from  each 
offeror.  Two  awardees  provided  two  sets  of  legitimate  resumes  that  satisfied  the 
professional  definition  of  an  investment  banker.  However,  each  of  the  two  awardees 
provided  resumes  for  consultants  for  their  last  (third  out  of  three)  investment  bankers. 
The  protester  deftly  utilized  a  dictionary  of  banking  to  argue  successfully  that  the 
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consultants  were  not  actually  investment  bankers.  GAO  held  that  the  agency  did  not 
evaluate  the  proposals  in  accordance  with  the  RFP.  The  question  at  hand  was  whether  or 
not  the  Investment  Bankers  had  to  have  experience  commensurate  with  their  title.  GAO 
held  that  experience  was  required.  GAO  cited  the  solicitation's  promise  that  the  agency 
would  assess  whether  "the  offer  [provided]  adequate  evidence  of  appropriate  academic 
credentials  and  depth  of  experience  and  professional  qualifications."  [Ref.  23]  The 
Comptroller  General  determined  that  this  promise  placed  the  offerors  on  notice  that  they 
must  include  in  their  proposals  three  resumes  of  individuals  who  had  worked  as 
investment  bankers.  The  third  resumes  of  the  two  awardees  clearly  did  not  contain 
personnel  with  experience  in  investment  banking.  The  implication  for  contracting 
officers  lies  in  the  need  to  explore  fully  the  requirement  for  which  a  solicitation  is  being 
crafted.  The  arguments  of  the  agency  indicated  that  it  did  not  need  three  experienced 
investment  bankers  to  satisfy  the  customer  needs.  However,  the  solicitation  called  for 
three  investment  bankers.  The  agency  overburdened  the  RFP  with  unnecessary 
requirements  and  failed  to  hold  two  awardees  to  these  requirements;  consequently,  AID 
was  unable  to  defend  two  of  their  contract  awards  to  the  satisfaction  of  GAO. 

2.  Follow  the  Solicitation 

As  in  previous  chapters,  a  Federal  agency's  inability  or  unwillingness  to  follow 
the  terms  of  solicitations  results  in  sustainments.  FYI  is  a  situation  in  which  the  protester 
was  the  only  offeror  to  meet  the  RFP  requirement  for  submission  of  personnel  data  files 


109 


(PDF)  for  50  percent  of  the  personnel  proposed  for  each  labor  category.  The  technical 
evaluation  team  (TET)  recognized  this  and  recommended  that  the  senior  selection  official 
(SSO)  award  the  contract  to  FYI.  The  SSO  was  swayed  only  by  the  awardee's  lower  cost 
and  ignored  the  TET's  recommendation.  The  SSO  believed  that  the  PDF  problem  could 
be  dealt  with  as  a  contract  administration  issue,  so  the  award  was  made  to  the  lower 
priced  proposal.  GAO  did  not  concur  and  held  that  the  awardee's  proposal  did  not  meet 
the  material  terms  of  the  solicitation.  The  holding  stemmed  from  a  previous  protest 
{National  Medical  Staffing,  Inc.;  PRS  Consultants,  Inc.,  69  Comp  Gen  500,  502  (1990)) 
where  the  Comptroller  General  had  held  that  "a  proposal  that  fails  to  conform  to  the 
material  terms  and  conditions  of  the  solicitation  should  be  considered  unacceptable  and 
may  not  form  the  basis  for  an  award."  [Ref.  28] 

GAO  also  determined  that  the  SSO's  intent  to  handle  the  PDF  problem  during 
contract  administration  was  undercut  by  a  provision  in  the  RFP  that  permitted  agency 
approval/disapproval  of  personnel  only  during  evaluation.    By  making  the  award  to 
RGII,  the  agency  accepted  the  individuals  that  were  proposed  in  an  unacceptable  manner 
(lacking  PDFs  for  50  percent  of  each  labor  category).  The  agency,  in  effect,  waived 
RGII's  personnel  qualifications  requirement,  which  resulted  in  unfair  and  unequal 
evaluation.  This  particular  holding  by  GAO  arose  from  a  "fundamental  principle  of 
federal  procurement  that  offerors  be  treated  equally;  that  is,  offerors  must  be  provided 
with  a  common  basis  for  the  preparation  of  proposals,  and  award  based  upon  the 
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requirements  stated  in  the  solicitation,  unless  the  offerors  are  notified  of  changes  in  (or 
relaxation  of)  the  agency's  stated  requirements."  [Ref.  28]  The  implication  for 
contracting  officers  is  that  the  RFP  must  be  followed.  All  provisions  must  be  observed  to 
ensure  fairness  to  the  offerors.  Any  changes  to  the  solicitation  must  be  announced  and 
made  available  to  all  offerors  to  preserve  fairness. 

Contracting  officer  should  take  these  two  valuable  lessons  with  them  to  avoid 
sustainable  protests  when  evaluating  labor  qualifications.    First,  contracting  officers  must 
ensure  the  solicitation  is  crafted  to  match  the  requirement.  Second,  adherence  to  the 
solicitation  is  critical  to  a  fair  award.  Chapter  VIII  addresses  improper  agency  pre-  and 
post-award  changes  that  resulted  in  protest  sustainments  in  1997. 
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VIII.    IMPROPER  AGENCY  PRE-  OR  POST-AWARD  CHANGE 

A.  INTRODUCTION 

Improper  Federal  agency  pre-  or  post-award  change  is  cited  as  a  sustaining 
argument  in  14  percent  of  the  successful  protests  in  1997.  Improper  changes  result  in  the 
unfair  evaluation  of  offerors'  proposals  pursuant  to  a  best  value  award.  A  change  of  a 
significant  nature  is  normally  followed  by  an  amendment  to  the  solicitation  (pre-award) 
or  modification  to  the  contract  (post-award).  The  three  protests  that  were  sustained  due 
to  improper  agency  changes  are  International  Data  Systems,  Inc.,  Marvin  J.  Perry  & 
Associates,  and  Symetrics  Industries  Inc.  This  chapter  discusses  the  FAR  requirements 
regarding  pre-  and  post-award  changes,  describes  the  three  sustained  protests,  and 
provides  analysis  and  conclusions. 

B.  CHANGES  AND  THE  FAR  PART  15  REWRITE 

The  fundamental  regulations  regarding  changes  in  Government  requirements  and 
changes  to  solicitations  are  addressed  in  the  FAR  Part  15  rewrite;  however,  there  were  no 
substantive  changes  from  the  previous  version.  The  basic  precepts  are: 

1 .  If,  either  before  or  after  receipt  of  proposals,  the  Government  changes  its 
requirements  or  terms  and  conditions,  the  contracting  officer  must  amend  the 
solicitation. 

2.  Amendments  issued  before  the  established  time  and  date  for  receipt  of 
proposals  must  be  issued  to  all  parties  receiving  the  solicitation. 
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3.  Amendments  issued  after  the  established  time  and  date  for  receipt  of  proposals 
must  be  issued  to  all  offerors  that  have  not  been  eliminated  from  the 
competitive  range. 

4.  If  a  proposal  of  interest  to  the  Government  involves  a  departure  from  the 
stated  requirements,  the  contracting  officer  must  amend  the  solicitation 
without  revealing  to  the  other  offerors  the  alternate  solution  or  any  other 
information  that  is  entitled  to  protection.  [Ref.  1:  15.206,  Ref.  13:  15.606] 

Post-award  contract  modifications  are  also  addressed  in  both  versions  of  FAR  Part 
1 5  with  no  significant  changes.  The  applicable  regulation  states  that  any  changes  to 
existing  contracts  are  required  to  be  in  scope  (minor  and  within  the  context  of  the  original 
contract).  [Ref.  13:  43.201]  Out-of-scope  changes  undermine  the  concept  of  fairness  in 
the  previous  best  value  evaluation  and  contract  award. 
C.         PROTESTS 

1.  International  Data  Systems,  Inc. 

International  Data  Systems,  Inc.,  B-277385,  Comptroller  General  of  the  United 
States,  8  October  1997,  (IDS)  was  discussed  in  Chapter  V  because  it  is  a  protest  that  was 
sustained  due  to  improper  agency  evaluation  of  technical  merit/ability.  IDS  was  also 
sustained  because  of  an  improper  agency  change  to  the  solicitation.  International  Data 
Systems  protested  the  Department  of  Interior  (DOI)  award  of  a  fixed-price,  indefinite 
delivery/indefinite  quantity  contract  for  personal  computers  with  Pentium  Pro  processors. 
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The  contract  time  period  was  to  span  six  months.  DOI  found  Applied  Computer 
Technology  (ACT)  to  be  the  offeror  representing  the  best  value  to  the  Government. 

The  RFP  stated  that  a  Technical  Evaluation  Panel  (TEP)  would  consider  the 
following  evaluation  criteria:  conformance  with  technical  specifications — 40  percent, 
past  performance — 40  percent,  and  price — 20  percent.  Forty-two  offers  were  received 
with  only  seven  in  the  competitive  range.  In  addition  to  the  Pentium  Pro  processors 
specified  in  the  solicitation,  ACT  proposed  a  newer  alternative:  the  Pentium  II  processor. 
ACT's  proposal  urged  the  agency  to  consider  the  Pentium  II  processor.  The  reason  cited 
by  ACT  was  that  the  RFP  stated  that  replacement  parts  and  support  must  be  available  for 
the  lifetime  of  the  system,  and  Intel  (the  processor  manufacturer)  had  indicated  that  the 
Pentium  Pro  would  be  phased  out  by  the  close  of  1997.    Intel  had  also  informed  ACT 
that  the  Pentium  II  was  the  scheduled  replacement  for  the  Pentium  Pro.  The  contracting 
officer  considered  the  alternative  Pentium  II  processor  offered  by  ACT  as  "state  of  the 
art."  [Ref.  21]    The  DOI  found  ACT's  proposal  to  be  the  best  value  and  awarded  them 
the  contract.  In  ACT's  contract,  the  agency  added  a  line  item  (for  the  Pentium  II 
processor)  that  did  not  appear  in  the  original  solicitation.  IDS  protested. 

IDS  argued  that  DOI  failed  to  amend  the  solicitation  properly.  The  Comptroller 
General  agreed.  GAO  stated  that  agency  requirements  changes  after  issuance  of  a 
solicitation  dictate  that  the  agency  must  amend  the  solicitation,  notify  offerors  of  the 
changes,  and  afford  them  an  opportunity  to  respond.  Furthermore,  the  requirement  to 
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amend  the  RFP  exists  even  after  the  submission  of  BAFOs,  up  to  the  time  of  award. 
GAO  determined  that  while  the  solicitation  outlined  a  Pentium  Pro  processor,  the  agency 
determined  during  the  course  of  the  procurement  that  the  Pentium  II  best  represented  the 
customer's  needs  and  should  have  been  incorporated  into  the  contract.  GAO  found  that 
DOI  was  required  to  amend  the  RFP  and  permit  all  offerors  an  opportunity  to  submit 
revised  proposals.  The  Comptroller  General  held  that  the  combination  of  the  improper 
agency  evaluation  of  technical  merit/ability  (Chapter  V)  and  the  failure  to  amend  the 
solicitation  prejudiced  IDS.  The  protest  was  sustained.  Since  the  computers  were  already 
delivered  and  accepted,  GAO  recommended  reimbursement  of  proposal  and  protest  costs 
to  IDS.  [Ref.  21] 

2.  Symetrics  Industries,  Inc. 

Sy metrics  Industries,  Inc.;  B-274246.3,  B-27 4246.4,  B-274246.5;  Comptroller 
General  of  the  United  States;  20  August  1997,  {Symetrics)  is  a  protest  where  a  significant 
quantity  change  resulted  in  sustainment.  Symetrics  Industries  protested  the  U.S.  Air 
Force  (US  AF)  award  of  a  fixed-price,  indefinite  delivery /indefinite  quantity  contract  for 
Lots  IV  through  VII  of  the  AN/ALE-47  Countermeasures  Dispenser  System  (CMDS). 
The  CMDS  was  an  electronic  warfare  system  used  by  the  Army,  Navy,  and  Air  Force  to 
guard  aircraft  from  hostile  missile  attacks.  The  contract  was  to  span  four  years.  USAF 
determined  Tracor,  Inc.  (Tracor)  to  be  the  offeror  whose  proposal  represented  best  value 
to  the  Government. 
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The  RFP  outlined  the  best  value  proposal  to  be  the  one  offering  the  best 
combination  of  utility,  technical  quality,  business  aspects,  risks,  and  price.    The 
evaluation  factors  were  identified  as  (descending  order  of  importance):  (1)  technical,  (2) 
schedule,  (3)  cost/price  (most  probable  life  cycle  cost),  and  (4)  management.    Tracor  had 
been  the  Government's  sole  source  for  CMDS  for  the  previous  eight  years.  Tracor 
demonstrated  in  its  proposal  that  it  was  the  only  offeror  who  could  best  produce  the  high 
quantity  of  CMDS  units  (Lots  IV  through  VII)  as  well  as  offer  an  accelerated  delivery 
schedule.  USAF  awarded  to  Tracor.  Symetrics  Industries  protested. 

The  protester  argued  that  the  evaluation  of  the  proposals  was  unreasonable 
because  it  was  based  on  an  erroneous  quantity  estimate.  The  protester  maintained  that  the 
requirement  for  sequencers  in  Lot  IV  (3,219  of  the  total  of  all  lots  combined  (3,775))  was 
so  unusually  high  that  only  Tracor  was  able  to  take  the  extraordinary  steps  to  meet  the 
agency's  need.  The  protester  did  not  dispute  the  original  evaluation  and  contract  award. 
However,  the  agency  canceled  Lot  IV  due  to  lack  of  funding  one  week  before  their 
request  for  BAFOs  and  failed  to  amend  the  solicitation.  Symetrics  asserted  that  many  of 
the  weakness  assessed  against  its  proposal  were  related  to  the  high  number  of  sequencers 
required  in  Lot  IV.  Because  the  original  solicitation  no  longer  reflected  an  accurate 
quantity  estimate,  Symetrics  asserted  that  the  best  value  tradeoff  in  the  final  evaluation 
lacked  a  reasonable  basis. 
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The  agency  countered  stating  that  (1)  the  funding  change  did  not  diminish  the  Air 
Force's  need  for  the  sequencers,  (2)  funding  could  have  been  provided  before  the  end  of 
the  lot  ordering  period,  and  (3)  the  protester  was  not  prejudiced  because  the  decision  not 
to  amend  the  solicitation  applied  to  all  offerors  equally.  GAO  stated  that  the  lot 
cancellation  represented  a  significant  change  to  the  Government's  requirement.  The 
evaluation  record  showed  that  only  Tracor  was  able  to  avoid  schedule  risk  and  meet  the 
agency's  need  regarding  Lot  IV.  Lots  V  through  VII  were  much  more  manageable 
because  they  totaled  only  491,  315,  and  503  respectively.  The  Comptroller  General 
stated  that  the  record  did  not  indicate  a  schedule  risk  problem  (for  Lots  V  through  VII) 
for  the  protester  in  light  of  the  reduced  requirements.  Furthermore,  the  prospect  of  future 
funding  (that  followed  the  cancellation)  did  not  impact  the  agency's  duty  to  amend  the 
solicitation.  As  a  result,  GAO  held  that  Symetrics  had  been  prejudiced  and  sustained  the 
protest.  The  Comptroller  General  recommended  reopening  of  the  competition,  issuance 
of  an  amended  solicitation,  and  reevaluation.  [Ref.  30] 

3.  Marvin  J.  Perry  &  Associates 

Marvin  J.  Perry  &  Associates;  B-277684,  B2 77685;  Comptroller  General  of  the 
United  States;  4  November  1997,  (MJP)  is  protest  in  which  a  post-award  change 
undermined  the  best  value  determination  and  resulted  in  sustainment.  Marvin  J.  Perry  & 
Associates  protested  the  Department  of  the  Navy  award  of  a  contract  for  563,  ten-piece 
sets  of  red  oak  sleeping  furniture  for  a  bachelor  enlisted  quarters  at  the  Naval  Training 


118 


Center.  Great  Lakes,  Illinois.  Nine  of  the  ten  types  of  furniture  were  available  from  the 
Federal  Supply  Schedule  (FSS).  The  FSS  program  is  a  General  Services  Administration 
(GSA)  program  that  provides  agencies  with  a  simplified  process  for  purchasing 
commonly  used  supplies  and  services  at  prices  associated  with  volume  buying.  The  tenth 
type  of  furniture  (footboards  for  beds)  was  solicited  in  conjunction  with  the  items  that 
were  listed  on  the  FSS.  The  Navy  determined  that  DCI's  proposal  represented  the  best 
value  to  the  Government. 

Before  the  procurement,  the  Navy  contacted  an  architectural  firm  to  design  the 
interior  to  the  enlisted  quarters.  The  firm  recommended  MJP  as  the  company  holding  the 
FSS  contract  offering  the  lowest  priced  red  oak  furniture  that  met  the  functional  aesthetic 
and  quality  needs  of  the  service.  The  Navy  solicited  quotes  from  other  FSS  vendors  of 
similar  products  to  ensure  that  MJP's  products  represented  the  best  value  to  the 
Government.  The  Navy  issued  a  Request  for  Quotations  (RFQ)  for  both  the  nine  types  of 
furniture  available  from  the  FSS  as  well  as  the  last  remaining  type  (footboards)  not  listed 
on  the  schedule.  DCI  proposed  the  lowest  quote  for  the  nine  items  at  $  557,127  while 
MJP  submitted  the  next  lowest  price  of  $  572,709.  However,  MJP  offered  the  lowest 
price  of  $  18,016  for  the  footboards,  and  DCI  quoted  the  next  lowest  price  of  $  19,789. 
The  Navy  awarded  the  contracts  to  the  offeror  with  the  lowest  total  price — DCI.  One 
month  before  the  delivery  date,  DCI  informed  the  Navy  that  their  supplier  had  mistakenly 
delivered  ash  wood  instead  of  the  red  oak.  DCI  proposed  that  the  agency  either  accept 
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the  furniture  in  ash  or  postpone  the  delivery  date.  After  reviewing  DCI  samples  of  ash 
wood  stained  to  match  the  appearance  of  red  oak,  the  Navy  decided  that  the  ash  furniture 
was  an  acceptable  substitute.  The  Navy  accepted  delivery  of  the  ash  furniture 
approximately  one  month  later.  Upon  learning  of  the  wood  substitution,  M  JP  protested. 

MJP  argued  that  changing  the  wood  from  red  oak  to  ash  was  unfair  to  other 
offerors.  MJP  maintained  that  they  based  their  quotes  on  the  prospect  of  providing 
furniture  made  from  the  superior  grade  of  wood  (red  oak)  and  that  they  had  not  been 
given  the  opportunity  to  offer  quotations  using  the  lower  quality  wood  (ash).  MJP 
pointed  out  that  authorization  to  use  ash  would  have  reduced  their  raw  wood  costs  by  40 
percent  and  their  overall  bid  price  by  over  $  47,000.  The  protester  bolstered  its  assertion 
with  copies  of  the  Weekly  Hardwood  Review  that  showed  that  ash  orders  at  the  time  sold 
for  $  525  per  board  foot  while  red  oak  sold  for  $  855  per  board  foot.  Finally,  the  protester 
stated  that  the  lack  of  prior  agency  authorization  to  change  the  wood  type  should  have 
caused  the  Navy  to  reject  DCI's  furniture  and  make  the  award  to  the  next  lowest  quoter— 
MJP. 

The  agency  first  argued  that  the  protest  addressed  a  contract  administration  issue 
and  was  not  under  the  purview  of  GAO.  The  Comptroller  General  stated  the  Navy's 
assertion  was  generally  true;  however,  GAO  asserted  that  out-of-scope  modifications  to 
existing  contracts  change  the  very  nature  of  the  original  contract  award.  For  this  reason, 
the  Comptroller  General  considered  the  protest.  The  Navy  also  stated  that  local 
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lumberyards  had  informed  them  that  the  scarce  nature  of  ash  resulted  in  only  a  one  to  five 
percent  reduction  in  price  from  red  oak.  GAO  was  not  swayed  and  stated  that  the  Navy 
had  not  presented  a  basis  for  the  Comptroller  General  to  question  the  protester's 
documented  contention  that  substantial  savings  was  to  be  had  through  the  wood 
substitution.  GAO  indicated  that  the  best  value  determination  was  undermined  and  stated 
that  the  "original  purpose  of  the  orders  was  so  substantially  changed  by  the  modification 
that  the  original  orders  and  the  modified  orders  are  essentially  different... By  not  issuing 
revised  RFQs  or  checking  other  vendors'  prices  for  ash  furniture,  the  agency  did  not 
ensure  that  it  received  the  best  price  for  the  ash  furniture,  which  was  the  stated  purpose 
for  issuing  the  RFPs  in  the  first  place."  Because  the  furniture  had  already  been  delivered, 
the  Comptroller  General  recommended  that  the  protester  be  reimbursed  for  its  protest 
costs.  [Ref.  29] 
D.         ANALYSIS  AND  CONCLUSIONS 

The  purpose  of  this  analysis  is  to  provide  contracting  officers  with  insight  as  to 
possible  agency  pitfalls  when  evaluating  labor  qualifications. 

1.  Contract  Awards  Cannot  be  Made  to  Offerors  Proposing  Departures 

from  Stated  Requirements  without  First  Amending  Solicitations 

IDS  is  a  protest  in  which  the  agency  discovered,  by  virtue  of  a  proposal,  that  they 
would  have  been  better  served  with  a  more  modern  computer  product.  The  awardee 
(ACT)  urged  the  use  of  Pentium  II  processors  rather  than  the  Pentium  Pro  processors 
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solicited  for  in  the  RFP.  The  agency  (DOI)  issued  the  contract  to  ACT  and  deviated  from 
the  solicitation  by  adding  a  new  line  item  to  procure  the  newer  Pentium  II  processors. 
GAO  held  that  to  the  extent  that  DOI's  needs  changed  to  include  the  Pentium  II 
processors,  the  agency  was  required  to  issue  an  amendment  permitting  all  offerors  a 
chance  to  submit  revised  proposals  on  a  common  basis.  The  holding  was  founded  in  both 
the  FAR  (15.606 — pre-rewrite)  and  a  previous  protest  (Symetrics — discussed  in  this 
chapter).  Contracting  officers  must  understand  that  when  a  proposal  brings  to  light  a 
different  need  for  the  Government,  the  need  for  an  amended  solicitation  also  emerges. 
[Ref.21] 

2.  Significant  Quantity  Changes  are  Also  Changes  to  Government 

Requirements  and  Require  Amendments  to  Solicitations 

Symetrics  shows  contracting  officers  that  quantity  changes  can  sometimes  drive  a 
need  for  amendments  to  solicitations.  The  USAF  wanted  to  procure  several  lots  of 
electronic  warfare  sequencers.  One  week  before  the  receipt  of  BAFOs,  the  program 
manager  canceled  Lot  IV  (3,219  sequencers)  leaving  only  Lots  V  though  VII  (cumulative 
total  of  1,309).  The  agency  failed  to  amend  the  solicitation  and  awarded  the  contract  to 
Tracor  (previous  sole  source  provider),  largely  because  they  had  the  ability  to  handle  the 
high  volume  and  fast  delivery  schedule  required  for  Lot  IV.  The  agency  maintained  that 
the  requirement  was  still  valid  and  that  funding  could  have  materialized.  GAO  found  that 
the  quantity  change  was  material  and  sustained  the  protest.  The  holding  was  founded  in 
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the  FAR  (15.606 — pre-re write)  stating  that  where  an  agency's  requirements  change  after 
a  solicitation  has  been  issued,  it  must  issue  an  amendment  to  notify  offerors  of  the 
changes.  GAO  continued  by  stating  that  one  circumstance  where  an  amendment  would 
be  required  is  a  significant  change  in  the  Government's  estimate  of  the  quantity  it  expects 
to  order.  Additionally,  the  Comptroller  General  considered  the  funding  issue.  GAO 
determined  that  when  an  agency  experiences  funding  reductions,  the  agency  is  required  to 
amend  the  solicitation  to  reflect  the  best  estimate  of  the  purchase  amount.  GAO  cited  a 
previous  protest  in  support  of  these  holdings  {Management  Systems  Designers,  Inc.,  B- 
244383.8,  Comptroller  General  of  the  United  States,  8  June  J  992).     Contracting  officers 
must  be  cognizant  of  funding  shortfalls  as  well  as  quantity  changes  and  ensure  amended 
solicitations  are  generated  when  significant  changes  occur.  [Ref.  30] 

3.  Out-of-Scope  Contract  Modifications  May  Result  in  Sustained 

Protests 

MJP  is  a  prime  example  of  Federal  agency  failure  to  recognize  an  out  of  scope 
modification.  The  Navy  sought  to  procure  red  oak  furniture  for  a  bachelor  enlisted 
quarters.  After  awarding  to  the  lowest  bidder,  the  agency  approved  a  change  to  in  wood 
type  because  the  awardee's  supplier  had  mistakenly  delivered  ash.  GAO  determined  that 
the  original  purpose  of  the  order  was  substantially  changed,  so  the  original  order  and 
modified  order  were  essentially  different.  There  are  several  bases  cited  by  GAO  in  this 
holding.  First,  the  out-of-scope  modification  took  a  perceived  contract  administration 
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issue  and  converted  it  to  a  protest  issue.  GAO  stated  protests  alleging  out-of-scope 
modifications  will  be  heard  by  the  Comptroller  General  because  such  actions  change  the 
nature  of  the  original  contract  award.  GAO  cited  a  previous  protest  in  its  explanation 
(Indian  Native  American  Employment  and  Training  Coalition,  64  Comptroller  General 
460,  1985).  Additionally,  GAO  stated  that  although  the  agency  conducted  this 
procurement  under  the  FSS  program,  it  chose  to  solicit  quotes  from  other  FSS  vendors. 
Having  held  a  competition  to  ensure  that  they  received  the  lowest  priced  item,  the  Navy 
was  obligated  to  ensure  that  the  competition  was  conducted  in  a  fair  manner.  The 
Comptroller  General  stated  that  the  FSS  did  not  exempt  an  agency  from  treating  vendors 
with  the  concern  for  fair  and  equitable  treatment  inherent  to  any  procurements.  GAO 
cited  several  previous  protests  to  support  this  point  (Haworth,  Inc;  Knoll  North 
American,  Inc.,  73  Comptroller  General,  1994;  SMS  Systems  Maintenance  Services,  Inc., 
B-270816,  Comptroller  General  of  the  United  States,  29  April  1996;  and  Dictaphone, 
Corp.,  B-254920.2,  Comptroller  General  of  the  United  States,  7  February  1994).  A  key 
issue  in  this  protest  is  fairness.  There  is  an  implicit  contract  between  the  Government  and 
vendors  that  all  offerors  will  be  treated  fairly  and  equitably  in  the  procurement  process. 
Post-award,  out-of-scope  modifications  violate  this  concept  and  invite  GAO  to  intervene 
if  protests  are  submitted.  This  is  precisely  why  GAO  heard  and  sustained  this  protest. 
Contracting  officers  should  take  extreme  care  to  ensure  modifications  are  within  scope. 
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Failure  to  do  so  can  bring  the  validity  of  a  best  value  evaluation  and  award  into  question. 
[Ref.  29] 

Contracting  officers  can  take  several  lessons  from  this  chapter.  First,  contract 
awards  cannot  be  made  to  proposals  offering  departures  from  stated  requirements  without 
first  amending  solicitations.  Second,  significant  quantity  changes  are  also  changes  to 
Government  requirements  and  require  amendments  to  solicitations.  Finally,  out-of-scope 
contract  modifications  may  result  in  sustained  protests.  Chapter  IX  provides  conclusions 
to  the  thesis. 
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IX.       FINAL  ANALYSIS  AND  CONCLUSION 
A.         INTRODUCTION 

The  purpose  of  this  chapter  is  to  provide  contracting  officers  with  an  overall 
picture  of  the  pitfalls  that  exist  during  the  solicitation  and  contract  award  process  when 
using  best  value  criteria.  First,  this  chapter  addresses  these  problems  within  the  context 
of  the  contracting  process.  Second,  the  reader  will  be  able  to  see  common  threads  of 
improper  agency  performance  that  resulted  in  protest  sustainments  in  1997.  Third,  the 
tradeoff  function  is  analyzed.  Fourth,  the  research  questions  are  formally  answered. 
Fifth,  the  chapter  conclusion  is  provided.  Finally,  areas  for  future  study  are  outlined. 

The  contracting  process  consists  of  six  steps  as  shown  below: 

1 .  Acquisition  Planning 

2.  Solicitation 

3.  Source  Evaluation 

4.  Negotiation 

5.  Contract  Award 

6.  Contract  Administration 

The  next  six  sections  of  this  chapter  discuss  the  sustained  protests  of  1997  within  the 
context  of  the  contracting  process. 
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B.         ACQUISITION  PLANNING 

Acquisition  planning  begins  upon  receipt  of  a  requirement  from  the  customer. 
"Acquisition  planning"  means  the  process  by  which  the  efforts  of  all  personnel 
responsible  for  an  acquisition  are  coordinated  and  integrated  through  a  comprehensive 
plan  for  fulfilling  the  agency  need  in  a  timely  manner  and  at  a  reasonable  cost.  It  includes 
developing  the  overall  strategy  for  managing  the  acquisition.   [Ref.  31:  7.101]  The 
method  of  contracting  and  type  of  contract  must  be  determined.  A  source  selection  plan 
and  statement  of  work  are  formulated.  Draft  requests  for  proposals  are  created  and  pre- 
solicitation  conferences  may  be  held.  Finally,  funding  must  be  verified. 

1.  Uncertainty  of  Requirement 

Barents  demonstrates  that  a  protest  sustainment  can  be  rooted  in  poor  acquisition 
planning.  The  protest  was  sustained  because  of  improper  agency  (AID)  evaluation  of 
labor  qualifications.  The  evaluation  of  labor  qualifications  were  found  to  be  improper 
because  two  of  the  awardees  failed  to  provide  three  resumes  for  personnel  that  met  the 
professional  definition  of  an  investment  banker.  GAO  held  that  AID  had  not  evaluated 
the  proposals  in  accordance  with  the  solicitation.  However,  the  source  of  the  problem  is 
evident  from  the  agency's  counter  argument  in  this  matter.  AID  maintained  that  despite 
the  RFP,  three  experienced  investment  bankers  were  not  necessary  to  satisfy  the  needs  of 
the  customer.  [Ref.  23]    Nevertheless,  the  solicitation  was  clear  in  its  call  for  the  three 
investment  bankers.  Effective  acquisition  planning  relies  on  an  accurate  relation  of  the 
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customer's  requirement  to  the  contracting  officer  so  that  the  contracting  process  may  be 
executed  properly.    In  Barents,  either  the  customer  misstated  its  need  or  the  agency 
misinterpreted  the  requirement.  Regardless,  the  protest  was  sustained  because  the  RFP 
mandated  requirements  that  were  not  necessary  for  the  customer's  needs  to  be  fulfilled, 
and  as  a  result,  the  agency  did  not  evaluate  the  proposal  in  accordance  with  the 
solicitation.  This  uncertainty  of  the  customer's  requirement  was  a  pivotal  breakdown  in 
acquisition  planning. 
C.         SOLICITATION 

Solicitations  are  used  in  negotiated  acquisitions  to  communicate  Government 
requirements  to  prospective  contractors  and  to  solicit  proposals.  RFPs  for  competitive 
acquisitions  must  describe  the  Government's  requirement,  anticipated  terms  and 
conditions  that  will  apply  to  the  contract,  information  required  to  be  in  the  offeror's 
proposal,  and  factors  and  significant  sub-factors  that  will  be  used  to  evaluate  the  proposal 
as  well  as  their  relative  importance.    [Ref.  1 :  15.203]  The  ability  to  craft  an  effective 
solicitation  is  critical  to  the  success  of  the  contracting  process  because  the  contracting 
officer  must  select  evaluation  criteria  to  be  followed  during  source  evaluation/selection. 
Poorly  formulated  solicitations  invariably  manifest  themselves  in  improper  agency 
actions  (evaluations  and  source  selections)  in  the  later  stages  of  the  contracting  process. 

HG  is  an  example  where  contradictory  requirements  in  the  solicitation  resulted  in 
an  improper  agency  evaluation  of  technical  merit.  The  evaluation  of  technical  merit  was 
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deemed  to  be  improper  because  the  solicitation  had  contradictory  requirements.  The  true 
requirement  of  the  agency  was  adequately  represented  in  a  conceptual  drawing  of  a 
proposed  office  for  the  Forest  Service.  The  drawing  contained  a  computer  room  within  a 
telecommunications  room.  However,  the  drawing  was  contradicted  by  an  overriding 
written  passage  dictating  that  the  computer  room  must  be  located  away  from  any 
telecommunications  equipment.  GAO  held  that  the  agency  did  not  follow  the 
solicitation;  however,  the  root  of  the  protest  stemmed  from  poorly  stated  requirements  in 
the  solicitation.  It  is  clear  in  HG  that  the  Forrest  Service  wanted  a  building  with  a  layout 
that  closely  resembled  the  conceptual  drawing.  Unfortunately,  the  drawing  did  not  match 
the  controlling  written  provision  for  distance  between  the  computer  room  and  the 
telecommunications  equipment.  The  agency's  inability  to  convey  their  needs  in  the 
solicitation  adequately  resulted  in  sustainment.  A  sloppily  written  solicitation  is  a  crucial 
mistake  threatening  the  successful  completion  of  the  contracting  process. 
D.         SOURCE  EVALUATION/SELECTION 

The  objective  of  source  selection  is  to  select  the  proposal  that  represents  the  best 
value  to  the  Government.  The  award  decision  is  based  on  an  evaluation  of  factors  and 
significant  sub-factors  that  are  tailored  to  the  acquisition  and  must  (1)  represent  the  key 
areas  of  importance  and  emphasis  to  be  considered  in  the  source  selection  decision  and 
(2)  support  meaningful  comparison  and  discrimination  between  and  among  competing 
proposals.  Proposal  evaluation  is  an  assessment  of  the  proposal  and  the  offeror's  ability 
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to  perform  the  prospective  contract  successfully.  [Ref.  1 :  15.302-15.305]  Most  of  the 
sustaining  arguments  in  1 997  were  rooted  in  poor  source  evaluation  and/or  source 
selection.  The  following  paragraphs  discuss  agency  improprieties  such  as  failure  to 
follow  the  solicitation,  failure  to  utilize  all  pertinent  facts,  improper  cost/price  evaluation, 
and  improper  changes  to  a  solicitation. 

1.  Failure  to  Follow  the  Solicitation  Consistently 

There  are  five  sustained  protests  where  the  Federal  agency  failed  to  follow  the 
solicitation  when  evaluating  proposals.  TSI  illustrates  improper  agency  evaluation  of 
technical  merit  resulting  in  the  Air  Force's  failure  to  follow  the  RFP.  The  awardee  had 
failed  to  provide  a  scheduling  system  as  prescribed  in  the  solicitation.  The  agency 
accepted  its  promise  to  provide  such  a  schedule  and  changed  the  awardee' s  adjectival 
score  from  the  lowest  to  the  highest  attainable.  The  schedule  never  materialized,  yet  this 
did  not  hinder  the  agency  from  awarding  to  OTE. 

In  a  related  protest  NavCom  demonstrated  how  an  improper  agency  evaluation  of 
past  performance  can  result  from  failure  to  follow  a  solicitation.  The  RFP  placed  great 
emphasis  on  "same  or  similar"  TACAN  experience.  A  peripheral  agency  goal  to  expand 
the  defense  base  in  the  TACAN  area  gave  the  contracting  officer  cause  to  deviate  from 
the  evaluation  criteria.  The  agency  judged  every  offeror's  past  performance  as  same  or 
similar;  however,  none  of  the  firms  had  TACAN  experience  except  the  protester. 
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Arora  is  an  example  of  improper  agency  evaluation  resulting  from  failure  to 
follow  a  solicitation  while  conducting  analyses  of  price  realism.  The  analyses  of  price 
realism  were  supposed  to  be  conducted  within  the  context  of  the  offerors'  past 
performance  evaluations.  However,  the  agency  (Navy)  was  so  concerned  with  Arora 
Group's  unusual  method  of  handling  wage  escalation,  they  rejected  the  protester's 
proposal  based  solely  on  perceived  price  realism  problems.  The  Navy  failed  to  analyze 
price  realism  within  the  context  of  past  performance  and  created  a  separate  and 
determinative  evaluation  factor  that  was  not  part  of  the  stated  evaluation  criteria.  GAO 
held  that  the  Navy  essentially  ignored  the  solicitation  requirements,  so  they  sustained  the 
protest. 

Tri-State  is  another  example  of  improper  agency  evaluation  of  price  resulting 
from  failure  to  follow  a  solicitation.  The  awardee  gained  a  price  advantage  by  improperly 
splitting  the  CLINs  and  offering  both  first  quantity  and  next  quantity  prices.  The  agency 
allowed  this  price  split  even  though  the  solicitation  had  no  provision  permitting  such  an 
action.  GAO  held  that  this  beneficial  pricing  strategy  was  not  afforded  to  all  offerors,  so 
the  contract  award  was  made  outside  the  parameters  set  forth  in  the  solicitation. 

Finally,  FYI  demonstrates  that  an  improper  agency  evaluation  of  labor 
qualifications  can  stem  from  RFP  deviation.  The  agency  technical  evaluation  team  (TET) 
recommended  award  to  the  protester.  The  senior  selection  official  ignored  the  TET  and 
awarded  to  the  lowest  cost  proposal  even  though  the  awardee  failed  to  meet  the  RFP 


132 


requirement  for  submission  of  personnel  data  files.    As  in  each  of  the  other  four  protests, 
GAO  held  that  the  agency  had  not  followed  or  held  the  awardee  to  the  requirements  listed 
in  the  solicitation. 

Deviation  from  the  RFP  is  the  most  common  misstep  regarding  source 
evaluation/selection,  and  it  transcends  the  various  sustainment  categories.  The  five 
protests  were  sustained  for  improper  evaluation  of  four  separate  tradeoff  elements 
(agency  evaluation  of  technical  merit,  past  performance,  price,  and  labor  qualifications); 
however,  the  root  of  each  sustainment  was  the  same:  failure  to  follow  the  solicitation.  The 
cause  may  sometimes  be  blamed  on  problems  that  occurred  earlier  in  the  contracting 
process  such  as  an  unclear  requirement  {Barents)  or  a  poorly  crafted  solicitation  (HG). 
However,  the  research  shows  that  there  is  more  often  a  problem  in  source  evaluation 
and/or  selection.  Contracting  officers  that  choose  not  to  adhere  to  solicitations  are  easy 
protest  victims.  Contracting  officers  that  inadvertently  stray  from  the  RFP  parameters 
fair  no  better  under  the  review  of  the  Comptroller  General.  Adherence  to  the  RFP  is 
crucial  to  successful  source  evaluation/selection  and  ultimately  the  successful  execution 
of  the  contracting  process. 

2.  Failure  to  Utilize  all  Relevant  Facts 

Agency  failure  to  utilize  all  facts  pertinent  to  a  source  evaluation/selection  was 
found  in  three  sustainments  for  improper  agency  evaluation  of  past  performance  and  one 
sustainment  for  improper  agency  evaluation  of  technical  merit.  The  past  performance 
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sustainments  are  MECSA,  STA  Engines,  and  IBSI.  In  MECSA,  fairness  and  consistency 
were  key  to  the  sustainment.  The  protester  was  downgraded  for  late  delivery  in  an 
otherwise  satisfactory  performance  appraisal.  Conversely,  the  awardee  was  not 
downgraded  for  safety  violations  in  a  previous  performance  appraisal  because  the  agency 
had  determined  that  the  "overall  assessment  had  been  satisfactory".  [Ref.  1 1]  This  was 
clearly  an  inconsistent  and  unfair  evaluation  of  past  performance.  PCC  chose  to  ignore 
the  past  performance  problems  of  the  awardee,  and  awarded  the  contract  to  FMS. 

IBSI  also  involved  an  agency  that  knowingly  disregarded  relevant  information 
about  a  protester.  In  addition  to  improperly  assigning  a  good  past  performance  rating  to 
the  awardee  (who  had  no  past  performance),  an  employee  at  the  agency  (VA)  had 
forgotten  to  fill  out  a  reference  form  and  reduced  the  protester's  pool  of  positive 
references  from  two  to  one.  The  contracting  officer  had  a  special  relationship  with  the 
protester.  This  same  contracting  officer  had  written  and  administered  the  contract—about 
which  the  information  was  misplaced.  Still  the  agency  failed  to  consider  the  information. 
GAO  held  that  the  information  was  clearly  known  to  the  contracting  officer  and  should 
have  been  included. 

STA  Engines  involved  agency  problems  of  a  different  nature.  The  agency 
(USCG)  erroneously  applied  irrelevant  past  performance  data  about  an  affiliate  and 
unreasonably  penalized  the  protester.  GAO  determined  that  the  actions  of  the  protester's 
affiliate  had  no  bearing  on  its  ability  to  perform  the  terms  of  the  anticipated  contract.  The 
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affiliate's  past  performance  was  stated  to  be  important  only  if  the  relationship  between 
the  offeror  and  affiliate  was  such  that  the  actions  of  one  party  directly  affected  the  ability 
of  the  other  party  to  perform. 

Cygnus  is  an  example  of  improper  agency  evaluation  of  technical  merit  stemming 
from  a  failure  to  read  the  protester's  BAFO  thoroughly.  The  protester  had  originally 
forgotten  to  include  a  statement  of  potential  contract  problems  and  solutions  in  the 
proposal.  The  required  information  was  included  in  the  protester's  BAFO,  but  the  agency 
did  not  factor  the  information  in  the  final  evaluation.  GAO  held  that  the  relevant 
information  should  have  been  evaluated.  In  all  four  cases,  the  Comptroller  General 
determined  that  an  improper  agency  evaluation  had  occurred  because  of  the  agencies' 
failure  to  assemble,  interpret,  and  utilize  relevant  information  properly  about  awardees 
and/or  offerors. 

In  each  of  the  three  past  performance  improprieties,  there  is  one  driving  theme: 
relevance  of  information.  If  past  performance  information  is  relevant  to  an  offeror's 
ability  to  satisfy  the  terms  of  a  contract,  it  must  be  factored  into  the  source  evaluation  and 
selection.  If  it  is  irrelevant,  it  must  be  withheld  from  evaluation.  Contracting  officers 
must  carefully  screen  and  interpret  all  contractor  information  to  determine  if  it  is  relevant 
to  the  contemplated  contract.  In  Cygnus  (improper  evaluation  of  technical  merit),  there 
was  no  question  of  relevance—only  a  lack  of  attention  to  detail  by  the  contracting  agency 
when  evaluating  technical  merit.  The  implication  for  the  contracting  officer  is  that  proper 
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use  of  information  is  key  to  fair  and  impartial  source  evaluation  and  selection.  It  is 
fundamental  to  the  successful  completion  of  the  contracting  process  in  pursuit  of  a  best 
value  award. 

3.  Remember  the  Importance  of  Cost/Price 

Cost/Price  is  the  central  and  most  important  tradeoff  element.  Contracting 
officers  cannot  forget  that  total  cost/price  must  be  evaluated  in  every  negotiated 
procurement.  Use  of  other  cost/price  related  data  (e.g.  cost  risk)  may  prove  to  be  helpful 
in  source  evaluation  and  selection;  however,  the  contracting  officer  is  never  relieved  of 
the  requirement  to  evaluate  and  compare  total  cost/price.  Boeing  and  Sylvest  are  two 
examples  where  agency  failure  to  evaluate  total  cost/price  resulted  in  sustainment.  In 
Boeing,  the  Federal  agency  (SOCOM)  was  unfamiliar  with  the  requirement  to  evaluate 
and  compare  total  cost  even  though  their  solicitation  clearly  and  correctly  dictated  this 
action.  Sylvest  involved  a  less  obvious  miscue.  The  agency  (TV A)  was  found  to  have 
evaluated  and  compared  the  total  prices  inadequately  because  they  awarded  the  contract 
to  a  firm  that  had  listed  a  necessary  computer  hard  disk  modification  as  optional.  GAO 
held  that  by  not  including  the  price  of  the  option  in  the  price  evaluation,  TVA  failed  to 
compare  the  awardee's  total  price  to  the  protester's  total  price.  Evaluation  and 
comparison  of  offerors'  total  cost/price  is  a  fundamental  precept  in  source  evaluation  and 
selection.  A  protest  that  establishes  a  Federal  agency  failure  of  this  type  is  very 
sustainable. 
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Another  important  cost/price  issue  is  effective  evaluation  of  cost  or  price  realism. 
Barents,  Geo,  and  Arora  all  demonstrate  that  agency  failure  in  this  regard  may  result  in 
sustainments.  In  Barents,  the  agency  cost  realism  analysis  was  found  to  be  inadequate 
because  it  failed  to  uncover  an  improper  contingency  placed  upon  the  awardee's  cost 
proposal.  In  Geo,  cost  and  price  realism  analyses  were  required.  The  agency  simply  did 
not  conduct  price  realism  analyses  on  any  of  the  proposals  or  BAFOs.  Regarding  cost 
realism,  an  agency  misconception  about  the  relationship  between  sample  task  costs  and 
task  order  requirements  contracts  doomed  the  evaluation.  The  solicitation  identified 
sample  task  cost  analyses  as  measures  of  contractor  understanding  only.  GAO 
determined  that  sample  task  costs  were  the  only  reasonable  method  for  cost  estimation  in 
such  a  contract  because  the  actual  costs  to  the  Government  would  certainly  have  varied 
according  to  the  issuance  of  task  orders. 

Arora  shows  that  an  erroneous  agency  analysis  of  a  protester's  price  realism 
results  in  sustainment.    The  protester  was  downgraded  because  the  Navy  had  concerns 
about  its  failure  to  provide  for  wage  escalation  in  its  price  proposal.  The  protester  had 
explained  that  its  long-term  employment  agreements  and  a  budget  for  recruitment  in  their 
General  and  Administrative  costs  precluded  the  need  for  wage  escalation  provisions. 
GAO  agreed  and  held  that  the  agency's  analysis  of  price  reasonableness  unreasonably 
discounted  the  labor  agreements  and  erroneously  undervalued  the  G&A  budget 
(recruitment)  by  90  percent.  It  is  very  important  for  contracting  officers  to  maintain  an 
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awareness  of  cost/price  issues.  Failure  to  evaluate  and  compare  total  cost  and  improper 
cost/price  realism  analyses  properly  was  a  sustainable  argument  in  five  of  the  21 
successful  protests.  As  a  result,  the  source  evaluation/selection  was  compromised  and  the 
contracting  process  disrupted. 

4.  Amend  Solicitations  to  Reflect  Changes  in  Government  Requirements 

Improper  changes  result  in  an  unfair  best  value  evaluation  and  source  selection. 
IDSI  and  Symetrics  are  good  examples  of  agency  failure  to  amend  a  solicitation  to  reflect 
changes  in  Government  requirements.  In  IDSI,  the  awardee  had  proposed  an  alternative 
state-of-the-art  computer  processor  (Pentium  II)  in  addition  to  the  one  for  which  the 
solicitation  was  issued  (Pentium  Pro).  The  resultant  contract  for  delivery  of  computers 
with  the  Pentium  II  processor  was  a  deviation  from  the  stated  requirements  in  the 
solicitation.  No  amendment  to  the  solicitation  had  been  issued,  and  the  result  was  a 
protest  sustainment.  Symetrics  demonstrates  that  a  significant  quantity  change  is  a 
change  to  the  Government  requirement  and  requires  an  amendment  to  the  solicitation. 
One  week  before  BAFOs,  funding  was  canceled  for  the  first  lot  in  the  contract.  This  lot 
represented  71  percent  of  the  total  contracted  quantity.  The  agency  felt  an  amendment  to 
the  solicitation  was  not  necessary  because  of  its  continued  need  for  the  goods  and  a 
potential  for  increased  funding.  GAO  stated  that  potential  funding  did  not  change  the  fact 
that  the  quantity  was  significantly  changed,  and  an  amendment  was  required.  The 
implication  for  contracting  officers  is  that  significant  changes  to  Government 
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requirements  (that  occur  up  to  the  point  of  award)  must  be  reflected  in  amendments  to 

solicitations.  This  is  to  ensure  fairness  for  all  offerors.  The  penalty  for  award  without  an 

amendment  is  an  improper  source  evaluation  and  selection  that  cripples  the  contracting 

process. 

E.         NEGOTIATIONS/DISCUSSIONS 

Negotiations  are  exchanges  that  occur  after  the  establishment  of  the  competitive 
range.  They  are  undertaken  with  the  intent  of  allowing  offerors  to  revise  their  proposals. 
Discussions  are  negotiations  tailored  to  each  offeror's  proposal  and  if  conducted,  must  be 
held  with  each  offeror  in  the  competitive  range.  The  purpose  of  discussions  is  to 
maximize  the  Government's  ability  to  obtain  the  best  value  based  on  the  requirement  and 
the  evaluation  factors  set  forth  in  the  solicitation.  [Ref.  1 :  15.306]  Failure  to  hold 
meaningful  discussions  was  prevalent  in  several  protests  in  1997. 

The  issue  of  discussions  is  key  to  five  of  the  21  of  the  protest  sustainments. 
Although  this  issue  transcends  the  various  tradeoff  elements,  contracting  officers 
soliciting  for  best  value  awards  seemed  to  have  difficulty  with  discussions  pertaining  to 
past  performance  and  cost/price  in  1997.  Three  of  the  protests  in  question  were  sustained 
in  part  because  of  an  agency  failure  to  hold  meaningful  discussions  about  past 
performance.  Two  protest  sustainments  involved  discussions  regarding  cost/price 
information. 
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Discussions  regarding  poor  past  performance  reports  must  be  held  to  give  offerors 
an  opportunity  to  account  for  the  situation.  ACI  illustrates  that  even  though  the  FAR  is 
clear  in  this  regard,  mistakes  can  still  be  made.  The  agency  mentioned  only  one  of  two 
negative  past  performance  reports  in  discussions.  Therefore,  the  discussions  were  not 
meaningful,  and  the  protest  was  sustained.  McHugh  shows  that  the  requirement  to  hold 
discussions  applies  to  internal  agency  references  as  well  as  third  party  reports.  The 
agency  maintained  that  intra-agency  reports  were  not  subject  to  interpretation,  so 
discussions  were  not  required.  GAO  held  that  the  FAR  makes  no  such  distinction  and 
sustained  the  protest.  STA  Engines  warns  of  the  danger  of  blanket  use  of  an  affiliate's 
past  performance  as  an  indication  of  an  offeror's  ability  to  meet  the  terms  of  a  contract. 
The  Federal  agency  improperly  applied  the  poor  past  performance  of  the  protester's 
affiliate.  In  addition,  they  failed  to  inform  the  protester  of  its  intent  to  use  the  affiliate's 
poor  standing  as  an  indication  of  their  ability  to  perform.  Again,  a  sustainment  occurred, 
in  part,  because  the  agency  failed  to  hold  meaningful  discussions  about  a  poor  report. 

Regarding  cost/price  evaluation,  Tri-state  demonstrates  that  contracting  officers 
must  understand  the  difference  between  clarifications  and  discussions.  When  discussions 
are  held,  they  must  be  held  with  all  offerors  in  the  competitive  range.    The  awardee  had 
proposed  a  different  (unauthorized)  pricing  scheme  by  splitting  the  CLINs.  In  addition, 
the  contracting  officer  wanted  to  clarify  whether  the  next  quantity  ordered  price  would 
apply  to  future  quantities.  The  Comptroller  General  held  that  they  agency's  perceived 
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clarifications  (about  the  price  of  follow  on  quantities)  were  actually  discussions.  The  fact 
that  these  discussions  were  not  held  with  all  offerors  contributed  to  the  sustainment  of 
this  protest.    In  Tri-State,  GAO  appeared  to  draw  a  firm  distinction  between  clarifications 
and  discussions.  This  effort,  in  the  pre-rewrite  months  of  1997,  may  have  been  designed 
to  clearly  define  these  concepts  before  the  rewrite  introduced  the  newest  form  of 
exchange—communications.  Boeing  is  a  protest  where  the  agency  first  failed  to 
accurately  conduct  cost  evaluation  (labor  hours  and  overhead)  and  then  failed  to  inform 
the  protester  during  discussions.    GAO  held  in  this  protest  that  the  errors  in  cost 
evaluation  were  not,  by  themselves,  sustainable  miscues.  However,  the  evaluation  errors 
coupled  with  the  failure  of  the  agency  to  raise  the  perceived  problems  during  discussions 
resulted  in  sustainment. 

Discussions  are  not  always  the  major  flaw  in  a  protest.  Sometimes  the  lack  of 
discussions  simply  accompanies  an  agency  impropriety  in  evaluation  of  one  of  the 
tradeoff  elements.  The  key  behind  meaningful  discussions  is  that  if  properly  held, 
protests  may  be  prevented.  In  STA  Engines  it  is  quite  possible  that  if  the  protester  had 
been  informed  of  the  agency's  application  of  its  affiliate's  past  performance,  objections 
by  the  protester  could  have  caused  agency  to  research  the  issue  further  and  make  proper 
source  evaluation  decisions.  Tri-State  is  another  example  where  discussions  may  have 
uncovered  the  awardee's  unfair  pricing  scheme  and  caused  the  agency  to  extend  the 
benefits  of  the  scheme  to  all  offerors.  In  Boeing,  discussions  may  have  resulted  in 
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correction  of  the  agency's  errors  during  cost  evaluation.  Such  actions  would  have  made 
these  contract  awards  valid.  The  implication  for  contracting  officers  is  that  the  FAR  Part 
1 5  rewrite  encourages  even  more  exchanges  between  agencies  and  offerors  than  were 
permitted  under  the  former  regulation.  Contracting  officers  must  make  use  of  the 
increased  opportunity  for  communications,  clarifications,  and  discussions  to  eliminate 
situations  such  as  ACI,  McHugh,  and  STA  Engines  where  discussions  simply  were  not 
held.  However,  Th-State  demonstrates  that  contracting  officers  must  be  fully  aware  of 
the  types  of  exchanges  they  are  utilizing  to  prevent  misapplication  of  clarifications, 
discussions,  or  communications.  Highly  trained  contracting  officers  will  be  able  to 
effectively  use  the  expanded  exchanges  to  ensure  fair  and  impartial  source 
evaluation/selection  and  proper  best  value  award. 
F.         CONTRACT  AWARD 

Contract  award  is  made  to  the  offeror  whose  proposal  represents  the  best  value  to 
the  Government.  One  situation  that  undermines  a  best  value  award  is  a  lack  of  supporting 
narrative  for  the  award.  This  problem  seems  to  be  rooted  in  evaluations  of  technical 
merit.  Three  GAO  protest  decisions  indicated  that  a  lack  of  narrative  contributed  to 
sustainment,  and  all  three  of  these  protests  were  sustained  due  to  improper  agency 
evaluation  of  technical  merit. 

In  MECSA,  the  agency  (PCC)  provided  no  justification  in  the  evaluation  record 
for  overriding  the  protester's  lower  price  when  making  award.  Also,  the  lack  of  narrative 
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provided  no  proof  that  PCC  had  accounted  for  a  special  certification  held  by  one  of  the 
protester's  subcontractors.  Jones  demonstrates  that  numerical  score  sheets  are  not 
enough  to  document  the  best  value  tradeoff.  The  protester  successfully  argued  that  the 
agency  could  not  justify  its  award  because  the  evaluation  record  consisted  only  of  score 
sheets  with  terse  bullet  narrative.  In  JW,  the  dearth  of  narrative  provided  no  explanation 
as  to  why  the  Federal  agency  scored  the  awardee  higher  under  an  evaluation  factor  when 
the  protester  appeared  to  be  significantly  superior.  The  only  supporting  documentation 
consisted  of  a  one  paragraph  per  offeror  narrative  that  failed  to  explain  any  of  the  scoring 
or  reasons  for  award. 

The  Government's  previous  preoccupation  for  low  cost/price  gave  rise  to  a 
situation  where  thorough  narrative  was  not  necessarily  critical  in  support  of  cost/price 
analyses,  matrices,  and  score  sheets.  In  best  value  contracting  a  tradeoff  occurs,  and  price 
is  now  only  one  of  several  significant  concerns.  It  is  important  to  note  that  narrative  is 
critical  because  the  benefit  of  a  tradeoff  is  not  self  evident  from  a  numerical  or  adjectival 
score.  The  implication  for  contracting  officers  is  that  best  value  contracting  drives  a 
higher  standard  for  supporting  narrative.  Neither  GAO  nor  the  FAR  Part  1 5  rewrite 
address  the  quantity  or  quality  of  narrative  required  to  adequately  support  a  best  value 
award.  In  MECSA,  Jones,  and  JW,  supporting  narrative  was  either  absent  or  barely 
existent.  The  best  value  tradeoff  is  difficult  to  express  numerically.  GAO  appears  to  be 
calling  for  contracting  officers  to  document  and  justify  their  award  thoroughly. 
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Supporting  comments  give  the  contracting  officer  a  chance  to  outline  the  rationale  for 
their  source  selection  fully.  Strong  award  decisions  are  built  on  a  foundation  of  strong 
evaluation  and  supporting  narrative.  Agency  awards  without  such  narrative  are  weak  and 
prone  to  sustainment. 
G.         CONTRACT  ADMINISTRATION 

Contract  administration  is  performed  by  either  the  PCO,  or  through  delegation, 
by  an  ACO  at  a  Contract  Administration  Office  (CAO).  The  overarching  theme  to 
contract  administration  is  oversight,  and  protests  are  often  thought  to  be  a  past  danger. 
However,  it  is  key  to  understand  that  protests  can  still  occur  through  action  or  inaction 
during  contract  administration.  FYI  illustrates  that  a  contracting  officer's  plan  to  address 
a  problem  through  contract  administration  may  result  in  an  improper  contract  award  and  a 
sustained  protest.  The  senior  selection  official  (SSO)  believed  that  the  awardee's  failure 
to  provide  personnel  data  files  (PDF)  for  50  percent  of  the  personnel  proposed  for  each 
labor  category  could  be  handled  through  a  substitution  clause  in  contract  administration. 
As  a  result,  the  SSO's  award  was  successfully  challenged  because  GAO  held  that  the 
awardee  had  not  met  the  requirements  of  the  solicitation. 

Clauses  allowing  the  Government  to  make  adjustments  in  the  post-award  phase  do 
not  relieve  the  contracting  officer  from  making  a  fair  and  impartial  best  value  award. 
MJP  demonstrates  that  out-of-  scope  changes  shift  the  very  foundation  of  a  best  value 
award  and  can  be  the  subject  of  a  protest.  The  agency  allowed  the  awardee  to  deliver 
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furniture  made  of  ash  instead  of  the  much  more  expensive  red  oak  called  for  in  the 
solicitation.  GAO  held  that  the  original  purpose  of  the  order  was  substantially  changed, 
and  the  original  order  and  modified  order  were  essentially  different.  The  best  value 
award  was  invalidated.  The  implication  for  contracting  officers  is  that  out  of  scope 
changes  can  convert  contract  administrative  issues  into  protest  issues.  GAO  will  hear 
protests  alleging  out-of-scope  changes  and  modifications.  In  this  situation,  the  ten  day 
protest  window  is  essentially  waived  by  GAO.  The  clock  begins  with  a  protesters  first 
knowledge  about  an  alleged  miscue.  A  post-award  agency  impropriety  and  subsequent 
protest  could  occur  months  or  years  after  award.  Provided  that  the  protester  files  the 
protest  in  a  timely  manner  (after  gaining  knowledge  of  an  alleged  offense),  GAO  will 
hear  the  protest.  In  sum,  Federal  agencies  must  understand  that  protests  can  be 
promulgated  by  a  contracting  officer's  actions  during  contract  administration.  Such 
actions  can  destroy  all  of  the  hard  work  that  goes  into  a  solid  and  proper  best  value 
award. 
H.        TRADEOFFS 

This  researcher  finds  no  evidence  that  the  act  of  making  tradeoffs  was  problematic 
for  contracting  officers  in  1997.  Because  tradeoffs  are  the  very  heart  of  a  best  value 
award,  one  may  have  assumed  that  problems  with  best  value  awards  could  be  traced  to 
improper  tradeoffs  by  a  Federal  agency.  The  evidence  suggests  otherwise.  Improper 
agency  tradeoff  was  alleged  in  40  of  the  108  best  value  protests  in  1997.  Interestingly, 
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allegations  of  improper  agency  tradeoff  were  never  sustained.  The  implication  is  that  the 
Comptroller  General  supports  the  contracting  officers'  judgment  regarding  best  value 
tradeoffs  provided  that  they  ( 1 )  evaluate  the  various  tradeoff  elements  (cost/price, 
technical  merit,  past  performance,  labor  qualifications)  properly,  (2)  award  the  contract 
without  making  improper  pre-award  changes  to  the  solicitation,  and  (3)  do  not  make  out- 
of-scope  post-award  changes. 
I.  RESEARCH  QUESTIONS 

1.  Primary  Research  Question 

To  what  extent  does  the  exercise  of  Federal  agency  contracting  officer  judgment,  in  best 
value  source  selection,  result  in  a  higher  rate  of  GAO protest  sustainment  than  other 
methods  of  source  selection? 

As  stated  in  Chapter  III,  the  research  evidence  shows  that  21  of  the  108  best  value 
protests  were  sustained  rendering  a  sustainment  rate  of  19.44  percent.  GAO's 
sustainment  rate  for  all  protests  in  1 997  was  1 2  percent. 

2.  Subsidiary  Research  Questions 

What  is  Best  Value,  the  genesis  of  the  concept,  the  evolution  of  this  type  of  contracting, 
current  implementation,  and  the  regulations  that  apply? 

As  stated  in  Chapter  II,  there  is  no  definitive  starting  point  for  the  concept; 
however,  businesses  have  historically  sought  to  purchase  goods  and  services  at  a  value  to 
facilitate  their  pursuit  of  profit.  The  Government  focus  on  best  value  began  with  the 
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1986  Packard  Commission  recommending  the  use  of  commercial  practices.  DoD 
officially  recognized  the  best  value  concept  in  a  study  entitled  Best  Value  Evaluation 
Process.  In  the  1990's,  acquisition  reform  measures  such  as  FAS  A  and  the  efforts  of  the 
Clinton  Administration  placed  best  value  squarely  at  the  center  of  Government 
contracting.  These  efforts  culminated  in  the  rewrite  of  the  FAR  Part  15  in  which  best 
value  was  defined  and  recognized  as  the  goal  of  every  Government  procurement. 

In  sum,  best  value  is  the  expected  outcome  of  any  acquisition  that  ensures  the 
customer's  needs  are  met  in  the  most  effective,  economical,  and  timely  manner.  It  is  the 
result  of  the  combination  of:  the  unique  circumstances  of  each  acquisition;  the  acquisition 
strategy;  choice  of  contracting  method;  and  the  award  decision.  Best  value  is  the  goal  of 
sealed  bidding,  simplified  acquisition,  commercial  item  acquisition,  negotiated 
acquisition,  and  any  other  specialized  acquisition  method  or  combination  of  methods. 
Through  the  best  value  continuum  the  Government  always  seeks  to  obtain  the  best  value 
in  negotiated  acquisitions  using  any  one  or  a  combination  of  source  selection  approaches, 
and  that  acquisition  should  be  tailored  to  the  requirement.  At  one  end  of  this  continuum 
is  the  lowest  priced-technically  acceptable  strategy  and  at  the  other  end  is  a  process  by 
which  elements  of  a  proposed  solution  can  be  traded  off  against  each  other  to  determine 
the  solution  that  provides  the  Government  with  the  overall  best  value.  All  such  tradeoffs 
must  be  conducted  according  to  the  source  selection  factors  and  sub-factors  identified  in 
the  solicitation.  [Ref.  3:  p.  1] 


147 


Can  the  reasons  for  the  sustainments  be  categorized?  If  so,  what  are  the  categories? 

As  stated  in  Chapter  III,  the  researcher  has  categorized  the  reasons  sustainment  by 
tradeoff  element.  The  categories  are  identified  by  protester  allegations  of  Government 
impropriety  in  their  agencies'  evaluation  of  the  following  tradeoff  elements:  cost/price, 
past  performance,  technical  merit,  and  labor  qualifications.  One  separate  category  has 
been  created  to  accommodate  improper  agency  pre-  and  post-award  changes  that 
invalidated  best  value  awards. 

Are  there  measures  that  a  contracting  officer  can  take  to  preclude  protests  regarding 
Best  Value  awards? 

Chapters  IV  through  IX  have  identified  lessons  contracting  officers  can  learn  from 
the  1 997  sustainments  to  avoid  improprieties  that  could  result  in  sustained  protests. 
These  lessons  include: 

•  The  need  for  contracting  officer  certainty  of  the  requirement  during 
acquisition  planning. 

•  The  importance  of  a  well-crafted  solicitation. 

•  The  critical  requirement  to  follow  the  solicitation  during  source  evaluation 
and  selection. 

•  The  need  to  use  all  relevant  facts  during  source  evaluation  and  selection. 

•  The  importance  of  cost/price  evaluations  and  cost/price  realism  analyses 
during  source  evaluation  and  selection. 


148 


•  The  dangers  of  failing  to  amend  a  solicitation  when  changes  in  Government 
requirements  occur  during  source  evaluation  and  selection. 

•  The  need  for  supporting  narrative  to  support  the  best  value  award. 

•  The  fact  that  post-award  out-of-scope  changes  can  result  in  sustainable 
protests. 

•  The  understanding  that  a  contracting  officer's  intent  to  fix  a  problem  with  a 
potential  awardee's  proposal  during  contract  administration  must  not  unfairly 
prejudice  the  other  offerors. 

How  may  contracting  officers  be  better  trained  regarding  best  value  awards? 

Contracting  officer  training  should  focus  on  the  traditional  importance  of 
cost/price  as  a  tradeoff  element;  however,  increased  emphasis  should  be  applied  to 
evaluation  strategies  and  techniques  for  the  other  tradeoff  elements  (past  performance, 
technical  merit,  and  labor  qualifications).  Furthermore,  contracting  officer  training  must 
enhance  awareness  that  pre-  and  post-award  changes  could  invalidate  an  otherwise  proper 
best  value  source  selection  and  result  in  a  protest  and  sustainment. 
J.         CONCLUSION 

It  is  in  the  basics  that  contracting  officers  failed  in  1 997.  Evaluations  of  the 
elements  of  tradeoff  were  often  improper  because  of  unsure  requirements,  lack  of 
discussions,  failure  to  use  relevant  facts,  failure  to  consider  total  price/cost,  etc.  It  is 
important  to  note  that  the  FAR  Part  15  rewrite  in  the  spirit  of  acquisition  reform  is 
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designed  to  create  a  smoother  path  to  contract  award  through  innovative  practices  such  as 
best  value  as  well  as  improving  quality  and  preserving  fairness.  However,  best  value  in 
some  ways  creates  a  higher  standard  for  the  contracting  officer.  Before,  the  focus  was 
simply  cost/price  or  related  factors.  Now  the  contracting  officer  must  deftly  evaluate  as 
many  as  four  tradeoff  elements  including  cost/price.  It  is  possible  that  the  requirement 
for  thorough  evaluation  of  multiple  tradeoff  elements  could  actually  increase  the  level  of 
the  contracting  officer's  workload  since  the  days  of  the  dominant  focus  on  cost/price.  In 
sum,  best  value  does  not  grant  the  contracting  officer  a  license  to  make  arbitrary  contract 
awards.  GAO  seems  to  be  consistent  in  their  approach  to  best  value  contracting. 
Contracting  officer  judgment  is  still  well  supported  by  GAO,  but  the  Comptroller 
General's  continuing  focus  on  fairness  and  impartiality  is  the  cornerstone  for  the  21 
sustainments.  Contracting  officers  must  be  mindful  of  this  fact  as  they  increase  their  use 
of  best  value  source  selection. 
K.         AREAS  FOR  FURTHER  RESEARCH 

The  following  areas  for  future  research  regarding  best  value  are: 

1 .  Conduct  follow-on  analyses  of  best  value  protests  for  years  beyond  1997. 

2.  Develop  a  model  to  assist  contracting  officers  when  engaged  in  best  value 
contracting. 
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3.  Conduct  an  in-depth  analysis  of  a  specific  tradeoff  element  (cost/price,  technical 
merit,  past  performance,  or  labor  qualifications)  as  it  relates  to  best  value 
contracting. 

4.  Conduct  an  in-depth  analysis  of  out-of-scope  changes  to  contracts  as  they  relate  to 
best  value  contracting. 

5.  Conduct  a  joint  analysis  of  best  value  contracting  with  GAO. 
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APPENDIX 


PROTESTS  OF  1997  INVOLVING  BEST  VALUE  CRITERIA 

Symbols:  S-Successfiil  Best  Value  Protest;  U-Unsuccessful  Best  Value  Protest; 


NC-Did  Not  Meet  Criteria  as  Best  Value 


Symbol  Protest  Name 


U       AAC  Associates,  Inc., 


Decision 


B-274928 


Date 


13-Jan-97 


NC  Aalco  Forwarding,  Inc. 

NC  ACS  Systems  &  Engineering,  Inc., 

NC  Advanced  Designs  Corporation 

U  Alcan  Environmental,  Inc., 

NC  All  State  Boiler  Work,  Inc., 

U  Alliant  Techsystems,  Inc., 


B-277241.12/.13        29-Dec-97 


U       Ameriko,  Inc. 


U       Ares  Corporation, 
S       Arora  Group,  Inc., 


B-275439.3 


B-275859.2 

B-277362 

B-276162/.2/.3 


NC  Allied  Signal,  Inc.,  Electronic  Systems,  B-275032/.2 

S  American  Combustion  Industries,  Inc.,  B-275057.2 

NC  American  Marketing  Associates,  Inc.,  B-274454.4 

U  American  Native  Medical  Trans.  L.L.C.,  B-276873 


B-277068 


NC      Analytical  &  Research  Technology,  Inc.,       B-276064 


B-275321/.2 
B-277674 


31-Mar-97 


B-275928.2/.3  21-Apr-97 


ll-Apr-97 

3-Oct-97 

2-May-97 

17-Jan-97 

5-Mar-97 

14-May-97 
5-Aug-97 

29-Aug-97 

7-May-97 

7-Feb-97 

lO-Nov-97 
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NC      Assets  Recovery  Systems,  Inc., 


B-275332 


NC      Atlantic  Scientific  &  Technology  Corp.  B-276334.2 


U  ATLIS  Federal  Services,  Inc., 

U  AVIATE  L.L.C., 

S  Barents  Group,  L.L.C., 

U  BE,  Inc.;  PAI  Corporation 

NC  Best  Foam  Fabricators,  Inc., 

S  Boeing  Sikorsky  Aircraft  Support, 

U  Booz- Allen  &  Hamilton,  Inc., 

U  Brisk  Waterproofing  Company,  Inc., 

U  Bristol-Myers  Squibb  Company, 

U  California  Environmental  Engineering, 

U  Canadian  Commercial  Corporation 

NC  Charles  E.  Smith  Companies 

U  Chek  F.  Tan  &  Company, 

NC  CHI  Fabrication  Services, 

U  Cincom  Systems,  Inc., 

NC  CitiWest  Properties,  Inc., 

NC  Command  Enterprises,  Inc., 


B-275065.2/.3 

B-275058.6/.7 

B-276082/.2 

B-277978/.2 

B-275436 

B-277263.2/.3 

B-275934.2 

B-276247 

B-275277 

B-274807/.2 

B-276945 

B-277391 

B-277163 

B-275079 

B-275055/.2 

B-274689.4 

B-277042 


NC     Commonwealth  Industrial  Specialties,  Inc.,   B-277833 


10-Feb-97 

27-Oct-97 

12-Feb-97 

14-Apr-97 

9-May-97 

16-Dec-97 

20-Feb-97 

29-Sep-97 

29-May-97 

27-May-97 

5-Feb-97 

3-Jan-97 

31-M-97 

25-Sep-97 

8-Sep-97 

22-Jan-97 

21-Jan-97 

26-Nov-97 

21-Aug-97 

25-Nov-97 
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U       Computer  Systems  Development  Corp.  B-275356 

U       Consolidated  Engineering  Services,  Inc.,       B-277273 


U  Cordev,  Inc., 

U  Creative  Apparel  Associates, 

U  Crown  Clothing  Corporation, 

U  Cubic  Applications,  Inc., 

S  Cygnus  Corporation, 

U  DIGICON  Corporation, 

U  Doss  Aviation,  Inc.;  Dominion  Av,  Inc. 

NC  Dynalantic  Corporation, 

U  DynaLantic  Corporation, 

U  Dynamic  Resources,  Inc., 

U  Eagle  Design  &  Management,  Inc., 

U  EastCo  Building  Services,  Inc., 

U  EBA  Engineering,  Inc.  , 

U  ECC  International  Corporation, 

U  ECG,  Inc., 

U  Ecology  and  Environment,  Inc., 


B-275596 

B-275139 

B-277505.2 

B-274768/.2/.3 

B-275181 

B-275060/.2 

B-275419/.2/.3 

B-274944.2 

B-274944.5 

B-277213/.2 

B-275062 

B-275334/.2 

B-275818 

B-277422/.2 

B-277738 

B-277061.2 


NC     ENDMARK  Corporation,  B-278139 

U       Environmental  Affairs  Management,  Inc.,      B-277270 


ll-Feb-97 
24-Sep-97 
7-Mar-97 
24-Jan-97 
31-Oct-97 

2-Jan-97 
29-Jan-97 
21-Jan-97 
20-Feb-97 
25-Feb-97 
25-Aug-97 
16-Sep-97 
21-Jan-97 
10-Feb-97 
31-Mar-97 
14-Oct-97 
20-Oct-97 

5-Sep-97 
31-Dec-97 
23-Sep-97 
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U  Environmental  Chemical  Corporation,  B-275819 

U  Environmental  Training&Consltg  Intl,  Inc.,  B-278185 

NC  Epoch  Engineering,  Inc.,  B-276634 

U  Federal  Computer  International  Corp.  B-276885 

NC  Fluid  Power  International,  Inc.,  B-278 1 1 2 

U  Food  Services  of  America,  B-276860 

S  For  Your  Information,  Inc.,  B-278352 

U  GEC-Marconi  Electronic  Systems  Corp.  B-276186/.2 


NC     General  Physics  Federal  Systems,  Inc. 


S       Geo-Centers,  Inc., 


U       Global  Associates,  Ltd., 


B-275934 


B-276033 


B-275534 


U       GTE  Hawaiian  Telephone  Company,  Inc.,     B-276487.2 


U  H.F.  Henderson  Industries, 

S  HG  Properties  A,  L.P., 

U  HSG-Holzmann  Technischer  Services 

U  HSG-SKE, 

U  IGIT,  Inc., 

U  Ingalls  Shipbuilding,  Inc., 

S  International  Business  Systems,  Inc., 

NC  International  Data  Products,  Inc. 


B-275017 

B-277572/.2/.3 

B-274992.2/.2 

B-274769/.3 

B-275299.2 

B-275830/.2/.3 

B-275554 


l-Apr-97 

5-Dec-97 

7-M-97 

29-Jul-97 

10-Dec-97 
28-M-97 

15-Dec-97 
21-May-97 

21-Apr-97 
5-May-97 
3-Mar-97 

30-Jun-97 
17-Jan-97 

29-Oct-97 

26-Feb-97 
6-Jan-97 

23-Jun-97 
7-Apr-97 
3-Mar-97 


B-275480.2/.3/.4  3-Apr-97 
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S       International  Data  Systems,  Inc., 


B-277385 


8-Oct-97 


S       J.A.  Jones  Management  Services,  Inc.,  B-276864 


U  Jack  Faucett  Associates, 

NC  JEOL  USA,  Inc., 

NC  Jet  Investments,  Inc., 

S  JW  Associates  Inc., 

U  JW  Associates  Inc., 

NC  L.A.  Systems,  Inc., 

U  Liebert  Federal  Systems,  Inc., 

U  Litton  Systems,  Inc.,  Amecom  Div. 

NC  Logicon  RDA, 

U  MAC's  General  Contractor, 

NC  Main  Building  Maint.,  Inc~Costs, 

U  Management  Resources,  Inc., 

S  Marvin  J.  Perry  &  Associates, 

U  Matrix  International  Logistics,  Inc., 

NC  MCA  Research  Corporation, 

S  McHugh/Calumet,  a  Joint  Venture, 

S  Mechanical  Contractors,  S.A. 

NC  Minotaur  Engineering, 


B-277555 

B-277160 

B-276215/.2 

B-275209 

B-275209.3 

B-276349 

B-274823 

B-275807.2 

B-276240/.2/.3 

B-276755 

B-260945.6 

B-276424/.2 

B-277684/5 

B-277208/.2 

B-276865 

B-276472 

B-277916 

B-276843 


24-Jul-97 

12-Sep-97 

2-Jul-97 

22-May-97 

30-Jan-97 

22-Jul-97 

9-Jun-97 

8-Jan-97 

16-Apr-97 

23-May-97 

24-Jul-97 

15-Dec-97 

13-Jun-97 

4-Nov-97 

15-Sep-97 

29-Jul-97 

23-Jun-97 

27-Oct-97 

22-May-97 
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U       MiTech,  Inc., 


B-275078 


23-Jan-97 


U  Motorola,  Inc., 

U  National  Office  Systems,  Inc., 

U  Navales  Enterprises,  Inc. 

S  NavCom  Defense  Electronics,  Inc., 

U  Nomura  Enterprise,  Inc., 

U  Northwest  Management,  Inc., 

U  Ouachita  Mowing,  Inc., 

NC  Pearl  Properties,  Inc., 

NC  Pemco  Aeroplex,  Inc—Recon.&Costs, 

U  Polar  Power,  Inc., 

U  PRB  Associates,  Inc., 

U  PRC,  Inc., 

NC  PRC,  Inc.— Reconsideration, 

U  Precision  Echo,  Inc., 

U  Quality  Elevator  Company,  Inc., 

U  Resource  Management  Intl,  Inc. 

U  Richard  M.  Milburn  High  School, 

NC  Rockville  Mailing  Service,  Inc., 

U  Rotair  Industries,  Inc., 


B-277862 
B-274785 
B-276122 
B-276163 
B-277768 
B-277503 
B-276075/.2 
B-277250.2 
B-275587.5/.6 
B-274846.2/.3 
B-277994/.2 
B-274698.2/.3 
B-274698.4 
B-276740/.2 
B-276750 
B-278108 
B-277018 
B-275073/.2 
B-276435.2 


3-Dec-97 

6-Jan-97 

13-May-97 

19-May-97 

19-Nov-97 

20-Oct-97 

8-May-97 

18-Sep-97 

14-Oct-97 

14-Jan-97 

18-Dec-97 

23-Jan-97 

10-Jul-97 

23-Jul-97 

23-M-97 

22-Dec-97 

19-Aug-97 

23-Jan-97 

15-M-97 
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NC      Rotary  Furnishing  Company, 


B-277704 


13-Nov-97 


B-274285.2/.3  19-May-97 


NC  Roxco,Ltd.,  B-277545  27-Oct-97 

U  Science  and  Engineering  Svcs.  Inc.,  B-276620  3-Jul-97 

U  SEEMA,Inc,  B-277988  16-Dec-97 

NC  Shel-Ken  Properties,  Inc.,  B-277250  18-Sep-97 

U  Siebe  Environmental  Controls,  B-275999.2  12-Feb-97 

S  ST  Aerospace  Engines  Pte.  Ltd.,  B-275725  19-Mar-97 

U  ST  Aerospace  Engines  Pte.  Ltd.,  B-275725.3  17-Oct-97 

U  Suddath  Van  Lines,  Inc. 

U  SWR,  Inc., 

S  Sylvest  Management  Systems  Corp. 

S  Symetrics  Industries,  Inc., 

NC  T.  Head  and  Company,  Inc., 

S  Technology  Services  International,  Inc.,  B-276506  21-May-97 

U  Tecom,  Inc.,  B-275518.2  21-May-97 

U  Telford  Aviation,  Inc.,  B-275896  16-Apr-97 

U  Terex  Cranes,  Inc.,  B-276380  10-Jun-97 

NC  Texstar,  Inc.,  B-275961/80  22-Apr-97 

U  The  Cube  Corporation,  B-277353  2-Oct-97 

U  The  Leader  Mortgage  Company,  B-274 110.2  19-May-97 


B-276878 


B-275935/.2 


29-M-97 


21-Apr-97 


B-274246.3/.4/.5       20-Aug-97 
B-275783  27-Mar-97 
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U  Theisinger  und  Probst  B-275756  25-Mar-97 

U  TMI  Services,  Inc.,  B-276624.2  9-Jul-97 

U  Trend  Western  Technical  Corp.  B-275395.2  2-Apr-97 

S  Tri-State  Government  Services,  Inc.,  B-277315.2  15-Oct-97 

U  UNICCO  Government  Services,  Inc.,  B-277658  7-Nov-97 

U  United  Ammunition  Container,  Inc.,  B-275213  30-Jan-97 

U  United  Terex,  Inc.,  B-275962.2  30-May-97 

U  USA  Electronics,  B-275389  14-Feb-97 

U  ViON  Corporation,  B-275063/.2  4-Feb-97 

U  Wastren,  Inc.,  B-276093  12-May-97 

NC  Wind  Gap  Knitwear,  Inc.,  B-276669  10-M-97 


160 


LIST  OF  REFERENCES 

1 .  Federal  Register,  Part  III,  Department  of  Defense,  General  Services  Administration, 
National  Aeronautics  and  Space  Administration,  48  CFR  Part  1  et  al..  Federal 
Acquisition  Regulation;  Part  15  Rewrite;  Contracting  by  Negotiation  and  competitive 
Range  Determination;  Final  Rule,  48  CFR  Chapter  1 ,  Federal  Acquisition  Regulation; 
Small  Entity  Compliance  Guide;  Final  Rule/Volume  62,  Number  189,  30  September 
1997. 

2.  The  Government  Contractor,  Volume  39,  Number  41,  An  Analysis  of  GAO's  Protest 
Decisions  on  Source  Selection,  1  513,  29  October  1997. 

3.  Highlights  of  the  FAR  Part  15  Rewrite,  Defense  Acquisition  Reform  Satellite 
Broadcast  Information,  http://www.acq.osd.mil/dau/arcc,  September  1997. 

4.  Commercial  Practices  for  Defense  Acquisition  Guidebook,  Defense  Systems 
Management  College,  Ft.  Belvior,  Virginia,  January  1992. 

5.  Zacks,  Michael  R.  and  Demas,  Thomas  A.,  Best  Value  Contracting,  Naval  Engineers 
Journal,  November  1 993 . 

6.  Acquisition  Reform  under  Federal  Acquisition  Streamlining  Act  of  1 994,  Volume  1 , 
The  Law,  Published  by  the  National  Contract  Management  Association. 

7.  Menker,  Janice,  Best  Value,  Debunking  the  Myth,  Program  Manager,  September- 
October  1992. 

8.  Bid  Protests  at  GAO:  A  Descriptive  Guide,  GAO/OGC-96-24,  Office  of  General 
Counsel,  Sixth  Edition,  1996. 

9.  The  Government  Contractor,  Volume  40,  Number  4,  GAO  Protests  Drop  By  18%, 
141,  28  January  1998. 

10.  Matter  of  International  Business  Systems,  Inc.,  B-275554,  Comptroller  General  of  the 
United  States,  3  March  1997. 

1 1 .  Matter  of  Mechanical  Contractors,  S.  A. .,  B-277916,  Comptroller  General  of  the 
United  States,  27  October  1997. 


161 


12.  Matter  ofNavCom  Defense  Electronics,  Inc.,  B-276163,  Comptroller  General  of  the 
United  States,  19  May  1997. 

13.  Federal  Acquisition  Regulation,  January  1997  (prior  to  the  October  1997  Part  15 
rewrite). 

14.  Matter  of  American  Combustion  Industries,  Inc.,  B-275057.2,  Comptroller  General  of 
the  United  States,  5  March  1997. 

15.  Matter  of  McHugh/Calumet,  A  Joint  Venture,  B-276472,  Comptroller  General  of  the 
United  States,  23  June  1997. 

16.  Matter  of  ST  Aerospace  Engines  Pte.  Ltd.,  B-275725,  Comptroller  General  of  the 
United  States,  19  March  1997. 

17.  Matter  of  J.  A.  Jones  Management  Services,  Inc.,  B-276864,  Comptroller  General  of 
the  United  States,  24  July  1997. 

18.  Matter  of  JW  Associates,  Inc.,  B-275209,  Comptroller  General  of  the  United  States, 
30  January  1997. 

19.  Matter  ofCygnus  Corporation,  B-275181,  Comptroller  General  of  the  United  States, 
29  January  1997. 

20.  Matter  ofHG  Properties  A,  L.P.,  B-277572,  B-277572.2,  B-277572.3,  Comptroller 
General  of  the  United  States,  29  October  1997. 

21.  Matter  of  International  Data  Systems,  Inc.,  B277385,  Comptroller  General  of  the 
United  States,  8  October  1997. 

22.  Matter  of  Technology  Services  International,  Inc.,  B276506,  Comptroller  General  of 
the  United  States,  21  May  1997. 

23.  Matter  of  Barents  Group,  L.L.C;  B-276082,  B276082.2;  Comptroller  General  of  the 
United  States;  9  May  1997. 

24.  Matter  of  Boeing  Sikorsky  Aircraft  Support;  B-277263.2,  B-27263.3;  Comptroller 
General  of  the  United  States;  29  September  1997. 
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25.  Matter  ofGeo-Centers,  Inc.,  B-276033,  Comptroller  General  of  the  United  States,  5 
May  1997. 

26.  Matter  ofSylvest  Management  Systems  Corporation;  B-275935,  B-275935.2; 
Comptroller  General  of  the  United  States;  21  April  1997. 

27.  Matter  o/Tri-State  Government  Services,  Inc.,  B-277315.2,  Comptroller  General  of 
the  United  States,  15  October  1997. 

28.  Matter  of  For  Your  Information,  Inc.,  B-278352,  Comptroller  General  of  the  United 
States,  15  December  1997. 

29.  Matter  of  Marvin  J.  Perry  &  Associates;  B-277684,  B277685;  Comptroller  General  of 
the  United  States;  4  November  1997. 

30.  Matter  ofSymetrics  Industries,  Inc.;  B-274246.3,  B-274246.4,  B-274246.5; 
Comptroller  General  of  the  United  States;  20  August  1997. 

31.  Federal  Acquisition  Regulation,  31  March  1998. 

32.  Matter  ofArora  Group,  Inc.,  B-277674,  Comptroller  General  of  the  United  States,  10 
November  1997. 
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INITIAL  DISTRIBUTION  LIST 


No.  Copies 

1 .  Defense  Technical  Information  Center 2 

8725  John  J.  Kingman  Rd.,  Ste  0944 

Ft.  Belvoir,  Virginia  22060-6218 

2.  Dudley  Knox  Library 2 

Naval  Postgraduate  School 

411  DyerRd. 

Monterey,  California  93943-5101 

3 .  Defense  Logistics  Studies  Information  Exchange 2 

U.S.  Army  Logistics  Management  Center 

Fort  Lee,  Virginia  23801 

4.  Dr.  David  V.  Lamm 5 

Naval  Post  Graduate  School 

Attn:  Code  SM/Lt 

Monterey,  California  93943-5000 


5.  Dr.  Mark  W.  Stone 

Naval  Post  Graduate  School 

Attn:  Code  SM/St 

Monterey,  California  93943-5000 

6.  Dr.  Sandra  M.  Desbrow 

Naval  Post  Graduate  School 

Attn:  Code  SM/Db 

Monterey,  California  93943-5000 

7.  Defense  Supply  Center  Richmond 

Attn:  LCDR  John  T.  Palmer,  SC,  USN 
8000  Jefferson  Davis  Highway 
Richmond,  Virginia  23297-5100 
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